kubernetes python api read_namespaced_pod_log returns http 400 with pod log as well in exception

Question

I'm trying to fetch pod logs using Kubernetes python library. I'm testing on minikube, I have added configuration.assert_hostname = False

API used is: read_namespaced_pod_log

Below is an example code:

#!/usr/bin/python3
import requests
import json
from kubernetes import client, config
from kubernetes.client import configuration
from kubernetes.stream import stream

config.load_kube_config()
configuration.assert_hostname = False
k8sapi = client.CoreV1Api()

resp_obj = k8sapi.list_pod_for_all_namespaces(
        watch=False,
        _preload_content=False
        )
pods_all = json.loads(resp_obj.data)


for pod in pods_all["items"]:
        pod_name  = pod["metadata"]["name"]
        pod_ns = pod["metadata"]["namespace"]
        pod_details = json.loads((k8sapi.read_namespaced_pod(name=pod_name, namespace=pod_ns, _preload_content=False)).data)
        for container in pod_details["status"]["containerStatuses"]:
                if "waiting" in container["state"]:
                        try:
                                pod_log = json.loads(k8sapi.read_namespaced_pod_log(name=pod_name, container=container, namespace=pod_ns, _preload_content=False).data)
                                #print(pod_log)
                                #print(pod_log)
                        except Exception as e:
                                #print("error_log: "+ str(e))
                                print("this is error")
                        #print(pod_log)
        print("-------")
        #print(pod_log)
                     

Exception error returned in response:

Traceback (most recent call last):
  File "/Users/gv/chatgpt_k8s/scripts/./chatgptk8s_monitor.py", line 26, in <module>
    pod_log = json.loads(k8sapi.read_namespaced_pod_log(name=pod_name, container=container, namespace=pod_ns, _preload_content=False).data)
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/api/core_v1_api.py", line 23747, in read_namespaced_pod_log
    return self.read_namespaced_pod_log_with_http_info(name, namespace, **kwargs)  # noqa: E501
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/api/core_v1_api.py", line 23866, in read_namespaced_pod_log_with_http_info
    return self.api_client.call_api(
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/api_client.py", line 348, in call_api
    return self.__call_api(resource_path, method,
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/api_client.py", line 180, in __call_api
    response_data = self.request(
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/api_client.py", line 373, in request
    return self.rest_client.GET(url,
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/rest.py", line 241, in GET
    return self.request("GET", url,
  File "/Users/gv/Library/Python/3.9/lib/python/site-packages/kubernetes/client/rest.py", line 235, in request
    raise ApiException(http_resp=r)
kubernetes.client.exceptions.ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Audit-Id': '9ea1de46-a89a-4c34-9ad3-c944f3ce9960', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Sun, 04 Jun 2023 19:06:36 GMT', 'Content-Length': '418'})
HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"container {\'name\': \'nginx\', \'state\': {\'waiting\': {\'reason\': \'ImagePullBackOff\', \'message\': \'Back-off pulling image \\"nginx:laterror\\"\'}}, \'lastState\': {}, \'ready\': False, \'restartCount\': 0, \'image\': \'nginx:laterror\', \'imageID\': \'\', \'started\': False} is not valid for pod nginx-deployment-cb99699bc-7jr98","reason":"BadRequest","code":400}\n'

Answer 1

In this call:

pod_log = json.loads(
    k8sapi.read_namespaced_pod_log(
        name=pod_name, container=container, namespace=pod_ns, _preload_content=False
    ).data
)

The value of the container parameter should be the name of the container in the pod for which you want logs, but in your code container is a structured variable created by this loop:

for container in pod_details["status"]["containerStatuses"]:

You probably want:

pod_log = json.loads(
    k8sapi.read_namespaced_pod_log(
        name=pod_name, container=container['name'], namespace=pod_ns, _preload_content=False
    ).data
)

There was a typo in my code, introduced because you're working with _preload_content=False and I generally do not.

Your except Exception as e is hiding error messages from you; this would have been obvious if you were to print out the exception message.

I've corrected the code. You'll find you have a new error, because the .data attribute returned by the read_namespaced_pod_log method does not contain JSON data; you'll probably want:

pod_log = k8sapi.read_namespaced_pod_log(
    name=pod_name, container=container["name"], namespace=pod_ns, _preload_content=False
).data