We run some automated tests against our app that's deployed locally (to a CI build agent) using k3s and skaffold.
How can we view (stdout or file) the pod logs when we run skaffold deploy --status-check?
(We've had errors during deployment that we can't see unless we ssh onto the build agent.)
The following, when run after the deployment was successful, captures logs, but this doesn't work if we run it before running skaffold deploy, presumably because it doesn't pick up any pods that are started after running this command:
kubectl logs \
--namespace x \
--follow \
--ignore-errors \
--max-log-requests 50 \
--all-containers \
--prefix \
--timestamps \
--selector release=y >"k8s.log" 2>&1 &
The output from skaffold deploy --status-check
looks something like this:
LAST DEPLOYED: Tue Oct 13 18:14:03 2020 NAMESPACE: smoke STATUS: deployed REVISION: 1 TEST SUITE: None Waiting for deployments to stabilize... - smoke:deployment/smoke-assets is ready. [7/8 deployment(s) still pending] - smoke:deployment/smoke-b is ready. [6/8 deployment(s) still pending] - smoke:deployment/smoke-c is ready. [5/8 deployment(s) still pending] - smoke:deployment/smoke-d: waiting for rollout to finish: 0 of 1 updated replicas are available... - smoke:deployment/smoke-e: waiting for rollout to finish: 0 of 1 updated replicas are available... - smoke:deployment/smoke-f: waiting for rollout to finish: 0 of 1 updated replicas are available... - smoke:deployment/smoke-g: waiting for rollout to finish: 0 of 1 updated replicas are available... - smoke:deployment/smoke-h: waiting for rollout to finish: 0 of 2 updated replicas are available... - smoke:deployment/smoke-d is ready. [4/8 deployment(s) still pending] - smoke:deployment/smoke-g is ready. [3/8 deployment(s) still pending] - smoke:deployment/smoke-e is ready. [2/8 deployment(s) still pending] - smoke:deployment/smoke-h: waiting for rollout to finish: 1 of 2 updated replicas are available... - smoke:deployment/smoke-h is ready. [1/8 deployment(s) still pending] 1/8 deployment(s) failed - smoke:deployment/smoke-f: running [kubectl --context k3d-smoke-tests-cluster rollout status deployment smoke-f --namespace smoke --watch=false] - stdout: "" - stderr: "error: deployment \"smoke-f\" exceeded its progress deadline\n" - cause: exit status 1 - smoke:deployment/smoke-f failed. Error: running [kubectl --context k3d-smoke-tests-cluster rollout status deployment smoke-f --namespace smoke --watch=false] - stdout: "" - stderr: "error: deployment \"smoke-f\" exceeded its progress deadline\n" - cause: exit status 1.
I found that stern can cope with new pods being added to the namespace.
So, running:
stern -n namespace . >my.log &
before the skaffold deploy does the trick.