"Kubernetes is an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure." - https://kubernetes.io/docs/whatisk8s

Glossary

More terms in the k8s glossary: https://kubernetes.io/docs/reference/glossary/

  • Container Network Interface (CNI) - https://github.com/containernetworking/cni
  • Container Runtime Interface (CRI) - https://github.com/containerd/cri/
  • Container Storage Interface (CSI) - https://github.com/container-storage-interface/spec
  • Custom Resource Definition (CRD)
  • Horizontal Pod Autoscaling (HPA)

cli usage

Learn about kubernetes

kubectl explain roles

Multiple kubeadm configs

The default config is ~/.kube/config, but if you want to use multiple configs you can do this:

export KUBECONFIG="${HOME}/code/kubespray/artifacts/admin.conf:${HOME}/.kube/config"

I have seen weird problems when the order of configs is changed, such as certificate-authority-data and client-certificate-data being missing.

kubeadm

"kubeadm: easily bootstrap a secure Kubernetes cluster." - kubeadm --help

Show your kubeadm tokens

$ sudo kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
ubyc9a.1eq2ihwtnz7c7c9e   23h       2018-05-24T16:19:33-04:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

See sudo kubeadm token -h for more usage.

kubectl

"kubectl controls the Kubernetes cluster manager." - kubectl --help

  • https://github.com/kubernetes/kubectl

  • kubectl get - show all resource types with short-hand versions.

  • kubectl completion -h - show how to configure completion for your shell.
  • kubectl config get-contexts - show which k8s configuration contexts you can control.
  • kubectl config use-context foo - switch to the foo context.
  • kubectl get nodes - show the nodes in the k8s cluster.
  • kubectl get pods - show deployed pods. there can be many pods per deployment.
  • kubectl get pods -n kube-system - show pods in a specific namespace.
  • kubectl get pods,hpa,deployment --all-namespaces - get several resource types at once, from all namespaces
  • kubectl describe pod foo
  • kubectl get deployment
  • kubectl describe deployment foo
  • kubectl get ns - show namespaces.
  • kubectl get pv - show physical volumes.
  • kubectl get svc -n kube-system - show a table of important details about running services in the kube-system namespace.
  • kubectl get pods -o yaml - show the yaml configs for the currently running status of every pod.
  • kubectl explain pods.spec - show documentation about pod specifications.
  • kubectl describe pods/echoserver - describe the pod whose Name is echoserver.
  • kubectl get rs - show replica sets.
  • kubectl expose deployment <deployment_name> --type=NodePort - create a service for the given deployment.
  • kubectl scale deployment <deployment_name> --replicas=5 - scale a deployment to 5 pods.
  • kubectl rollout history deployment <deployment_name>
  • kubectl get cm - get a list of config maps.

Working with several configs

Sometimes you want to have individual configs, such as when you are using configs that are updated by other engineers and pulled down via git, and sometimes you want to have one monolithic config, such as when you are using a tool that cannot easily work with multiple configs.

Merge several configs

This produces a monolithic file named kube_config which can be moved to ~/.kube/config. It merges the contents of your existing ~/.kube/config file.

REPO_DIR=/path/to/repo/
export KUBECONFIG="${HOME}/.kube/config"
for X in $(find "$REPO_DIR/kube_config.d" -name '*.config') ; do
    KUBECONFIG+=":$X"
done
kubectl config view --flatten > kube_config
echo "Config file successfully created at ${PWD}/kube_config"
echo "Run: mv -i ${PWD}/kube_config ${HOME}/.kube/config"

Create a KUBECONFIG env var from several config files

This produces a KUBECONFIG that looks like file1:file2:file3

REPO_DIR=/path/to/repo/
KUBECONFIG="${HOME}/.kube/config"
for config in $(find "$REPO_DIR/kube_config.d" -name '*.config') ; do
    KUBECONFIG+=":$config"
done
echo "KUBECONFIG=${KUBECONFIG}" ;

Watch what's going on in your cluster

watch kubectl get pods --all-namespaces -o wide

Show logs for a given pod since N hours ago

pod_name=httpbin
kubectl logs $pod_name --since=12h

The --since arg can take [s]econds, [m]inutes and [h]ours. Longer durations should use --since-time=<rfc3339 timestamp>

Show logs for a given pod since a given date

The --since-time arg takes RFC3339 datetime. EG: 1991-08-03T13:31:46-07:00. This format requirement is strict, and is incompatible with the GNU date --rfc-3339=seconds output, which uses a space instead of a T to separate the full date from the full time, and +%FT%F%z, which does not include a colon between hours and minutes.

pod_name=httpbin
kubectl logs $pod_name --since-time="$(date --iso-8601=seconds -d '-5 weeks')"

Output custom column names

$ kubectl get pvc --all-namespaces -o custom-columns='NAME:metadata.name,SIZE:spec.resources.requests.storage'
NAME                   SIZE
foo-logs               256Gi
test-volume-2          1Gi
some-awesome-service   5Gi

$ kubectl get pods -o custom-columns='NAME:.metadata.name,START_TIME:.status.startTime,.spec.containers[0].env[?(@.name == "GITLAB_USER_EMAIL")].value' | grep -E 'NAME|jobs'
NAME                                                 START_TIME             GITLAB_USER_EMAIL
runner-ppzmy1zx-project-11548552-concurrent-0q2pmk   2019-10-23T17:00:56Z   user2@example.com
runner-ppzmy1zx-project-11548552-concurrent-1f7nfx   2019-10-23T17:04:27Z   user1@example.com
runner-ppzmy1zx-project-11548552-concurrent-2n84rv   2019-10-23T17:04:19Z   user1@example.com

Installations

The standard way to install k8s by yourself is to use kubeadm.

Manually on Ubuntu 16

# as root
swapoff -a # https://github.com/kubernetes/kubernetes/issues/53533
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
echo "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable" > /etc/apt/sources.list.d/docker.list
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list
apt update
apt dist-upgrade -y
apt install -y apt-transport-https ca-certificates curl software-properties-common
apt install -y docker-ce
apt install -y kubelet kubeadm kubectl
kubeadm init

kubeadm init guide: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#instructions

DNS

Kubernetes lets you resolve resources via DNS

Enable k8s dns logging

kubectl -n kube-system edit configmap coredns
# Add 'log' to the 'Corefile' config

DNS Entity map

  • Kubernetes Service: <service>.<namespace>.svc.cluster.local (eg: httpbin.default.svc.cluster.local) sh kubectl get svc --all-namespaces -o jsonpath='{range .items[*]}{.metadata.name}{"."}{.metadata.namespace}{".svc.cluster.local\n"}' kubectl get svc --all-namespaces -o json | jq -r '.items[] | "\(.metadata.name).\(.metadata.namespace).svc.cluster.local"'
  • With core-dns you can run dig SRV +short *.*.svc.cluster.local to get a list of all serivces
  • Kubernetes service srv records: ....svc.cluster.local (eg: _http._tcp.httpbin.default.svc.cluster.local)
  • Pod by IP address: ..pod.cluster.local (eg: 192-168-1-6.default.pod.cluster.local)
  • Pod by name: ...?

crictl

crictl is a tool to inspect the local Container Runtime Interface (CRI)

user@k3snode:~$ sudo crictl pods
POD ID          CREATED       STATE  NAME                            NAMESPACE    ATTEMPT
688ecc2d9ce4d   2 weeks ago   Ready  log-fluentd-676d9d7c9d-ghz5x    default      8
ee1d8b0593e71   2 weeks ago   Ready  tiller-deploy-677f9cb999-rx6qp  kube-system  7
1153f4c0bd1f4   2 weeks ago   Ready  coredns-78fcdf6894-qsl74        kube-system  8
5be9c530c8cdc   2 weeks ago   Ready  calico-node-59spv               kube-system  10
d76d211830064   2 weeks ago   Ready  kube-proxy-cqdvn                kube-system  104
aa1679e0bfcca   2 weeks ago   Ready  kube-scheduler-s1               kube-system  10
ef64eea461bc0   2 weeks ago   Ready  kube-controller-manager-s1      kube-system  10
14ec5abe1e3ab   2 weeks ago   Ready  kube-apiserver-s1               kube-system  11
d4ce465a0942f   2 weeks ago   Ready  etcd-s1                         kube-system  10

Links