Kubernetes Usage
kubeconfig
AWS
- Ensure you have access to EKS.
- Login to the Bastion if the API Key is setup for public access
- Get the credentials
KUBECONFIG=./kubeconfig aws --profile=account eks update-kubeconfig --cluster cluster-name
- There can be multiple clusters so pick the correct cluster.
- Ensure that you set
export KUBECONFIG=./kubeconfig
to get the correct KUBECONFIG file. This can be added into you.bashrc
or.zshrc
List Running Pods
kubectl get pods --all-namespaces
Kubernetes lets you divide your cluster into namespaces. Each namespace can
have its own set of resources. The above command lists all running pods on
every cluster. Pods in the kube-system
namespace belong to Kubernetes and
helps it function.
Helm
"SSH"
To connect to the application look at the namespaces:
kubectl get pods --all-namespaces
kubectl exec -it -n <namespace> <pod> -c <container> -- bash
Logs
kubectl get pods --all-namespaces
kubectl logs -f -n <namespace> <pod> -c <container>
This lets you view the logs of the running pod. The container running on the pod should be configured to output logs to STDOUT/STDERR.
Describe Pods
Troubleshooting Pods
kubectl describe pods
Common Errors:
- OOMError
- CrashLoopBackup
- ImageNotFound
FAQ
How can I restart a pod?
If you pod is not responding or needs a restart the way to do it is to use the following command. This will delete the pod and replace it with a new pod if it is a part of a deployment.
kubectl delete pod <pod-name>
How can we remove pods?
This has to be done through the deployment in the helm chart. Another way to do it is to scale down
kubectl scale --replicas=0 -n <namespace> deployment/<deploymentname>
How can I add pods?
This has to be done through the deployment in the helm chart.
How can I add nodes?
Nodes are added through the Terraform module variable. Please check the infrastructure code and where the modules are defined. These will create the scaling groups for nodes.
How can I remove nodes?
Nodes are added through the Terraform module variable. Please check the infrastructure code and where the modules are defined. These will create the scaling groups for nodes.
How can I restart nodes?
Nodes can be terminated using the following script:
aws_profile=aws_profile
for i in $(kubectl get nodes | awk '{print $1}' | grep -v NAME)
do
kubectl drain --ignore-daemonsets --grace-period=60 --timeout=30s --force $i
aws --profile #{aws_profile} ec2 terminate-instances --instance-ids $(aws --profile $aws_profile ec2 describe-instances --filter "Name=private-dns-name,Values=#{node}" | jq -r '.Reservations[].Instances[].InstanceId'`)
sleep 300 # Wait 5 mins for the new machine to come back up
done