Kubernetes recipes: Maintenance and troubleshooting
You want to react based on the status of a resource—say, a pod—in a script or in another automated environment like a CI/CD pipeline.
Use kubectl get $KIND/$NAME -o json
and parse the JSON output using one of the two methods described here.
If you have the JSON query utility jq
installed, you can use it to parse the resource status. Let’s assume you have a pod called jump
and want to know what Quality of Service (QoS) class1 the pod is in:
$ kubectl get po/jump -o json | jq --raw-output .status.qosClass BestEffort
Note that the --raw-output
argument for jq
will show the raw value and that .status.qosClass
is the expression that matches the respective subfield.
Another status query could be around the events or state transitions:
$ kubectl get po/jump -o json | jq .status.conditions [ { "lastProbeTime": null, "lastTransitionTime": "2017-08-28T08:06:19Z", "status": "True", "type": "Initialized" }, { "lastProbeTime": null, "lastTransitionTime": "2017-08-31T08:21:29Z", "status": "True", "type": "Ready" }, { "lastProbeTime": null, "lastTransitionTime": "2017-08-28T08:06:19Z", "status": "True", "type": "PodScheduled" } ]
Of course, these queries are not limited to pods—you can apply this technique to any resource. For example, you can query the revisions of a deployment:
$ kubectl get deploy/prom -o json | jq .metadata.annotations { "deployment.kubernetes.io/revision": "1" }
Or you can list all the endpoints that make up a service:
$ kubectl get ep/prom-svc -o json | jq '.subsets' [ { "addresses": [ { "ip": "172.17.0.4", "nodeName": "minikube", "targetRef": { "kind": "Pod", "name": "prom-2436944326-pr60g", "namespace": "default", "resourceVersion": "686093", "uid": "eee59623-7f2f-11e7-b58a-080027390640" } } ], "ports": [ { "port": 9090, "protocol": "TCP" } ] } ]
Now that you’ve seen jq
in action, let’s move on to a method that doesn’t require external tooling—that is, the built-in feature of using Go templates.
The Go programming language defines templates in a package called text/template
that can be used for any kind of text or data transformation, and kubectl
has built-in support for it. For example, to list all the container images used in the current namespace, do this:
$ kubectl get pods -o go-template \ --template="{{range .items}}{{range .spec.containers}}{{.image}} \ {{end}}{{end}}" busybox prom/prometheus