Kubernetes Usage

kubeconfig

AWS

  • Ensure you have access to EKS.
  • Login to the Bastion if the API Key is setup for public access
  • Get the credentials
KUBECONFIG=./kubeconfig aws --profile=account eks update-kubeconfig --cluster cluster-name
  • There can be multiple clusters so pick the correct cluster.
  • Ensure that you set export KUBECONFIG=./kubeconfig to get the correct KUBECONFIG file. This can be added into you .bashrc or .zshrc

List Running Pods

kubectl get pods --all-namespaces

Kubernetes lets you divide your cluster into namespaces. Each namespace can have its own set of resources. The above command lists all running pods on every cluster. Pods in the kube-system namespace belong to Kubernetes and helps it function.

Helm

"SSH"

To connect to the application look at the namespaces:

kubectl get pods --all-namespaces
kubectl exec -it -n <namespace> <pod> -c <container> -- bash

Logs

kubectl get pods --all-namespaces
kubectl logs -f -n <namespace> <pod> -c <container>

This lets you view the logs of the running pod. The container running on the pod should be configured to output logs to STDOUT/STDERR.

Describe Pods

Troubleshooting Pods

kubectl describe pods

Common Errors:

  • OOMError
  • CrashLoopBackup
  • ImageNotFound

FAQ

How can I restart a pod?

If you pod is not responding or needs a restart the way to do it is to use the following command. This will delete the pod and replace it with a new pod if it is a part of a deployment.

kubectl delete pod <pod-name>

How can we remove pods?

This has to be done through the deployment in the helm chart. Another way to do it is to scale down

kubectl scale --replicas=0 -n <namespace> deployment/<deploymentname>

How can I add pods?

This has to be done through the deployment in the helm chart.

How can I add nodes?

Nodes are added through the Terraform module variable. Please check the infrastructure code and where the modules are defined. These will create the scaling groups for nodes.

How can I remove nodes?

Nodes are added through the Terraform module variable. Please check the infrastructure code and where the modules are defined. These will create the scaling groups for nodes.

How can I restart nodes?

Nodes can be terminated using the following script:

aws_profile=aws_profile

for i in $(kubectl get nodes | awk '{print $1}' | grep -v NAME)
do
        kubectl drain --ignore-daemonsets --grace-period=60 --timeout=30s --force $i
        aws --profile #{aws_profile} ec2 terminate-instances --instance-ids $(aws --profile $aws_profile ec2 describe-instances --filter "Name=private-dns-name,Values=#{node}" | jq -r '.Reservations[].Instances[].InstanceId'`)
        sleep 300 # Wait 5 mins for the new machine to come back up
done