I'm having a problem with JSON format produced by docker compose ps --format json command.
Here's my example docker-compose.yml file:
version: '3.5'
services:
nginx:
image: nginx
links:
- php
php:
image: php:fpm
I start my containers by docker compose up -d. When I run docker compose ps it produces the correct output:
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
test_docker-nginx-1 nginx "/docker-entrypoint.sh nginx -g 'daemon off;'" nginx 4 minutes ago Up 4 minutes 80/tcp
test_docker-php-1 php:fpm "docker-php-entrypoint php-fpm" php 4 minutes ago Up 4 minutes 9000/tcp
I wanted to customize the above output a little bit by using --format json and jq tool. However it appeared that docker compose ps --format json produces an incorrect output:
{"Command":"\"/docker-entrypoint.sh nginx -g 'daemon off;'\"","CreatedAt":"2023-09-20 14:32:55 +0200 CEST","ExitCode":0,"Health":"","ID":"956ee02dc6beb2b32fb15dac9cfb3be37bf09bcfd5fe05d9c23b31ba2fbc2108","Image":"nginx","Labels":"com.docker.compose.project.working_dir=/var/www/test_docker,com.docker.compose.version=2.21.0,com.docker.compose.config-hash=0483cc81f74ebb9392330ce8680a4d528986eecb650af475b34bdfe90bfa1dc3,com.docker.compose.container-number=1,com.docker.compose.depends_on=php:service_started:true,com.docker.compose.image=sha256:f5a6b296b8a29b4e3d89ffa99e4a86309874ae400e82b3d3993f84e1e3bb0eb9,com.docker.compose.project=test_docker,com.docker.compose.project.config_files=/var/www/test_docker/docker-compose.yml,maintainer=NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e,com.docker.compose.oneoff=False,com.docker.compose.service=nginx","LocalVolumes":"0","Mounts":"","Name":"test_docker-nginx-1","Names":"test_docker-nginx-1","Networks":"test_docker_default","Ports":"80/tcp","Publishers":[{"URL":"","TargetPort":80,"PublishedPort":0,"Protocol":"tcp"}],"RunningFor":"6 minutes ago","Service":"nginx","Size":"0B","State":"running","Status":"Up 6 minutes"}
{"Command":"\"docker-php-entrypoint php-fpm\"","CreatedAt":"2023-09-20 14:32:53 +0200 CEST","ExitCode":0,"Health":"","ID":"9ace40ff342457bceeff0433fd882cf65e819ac85cef4216df8815b8630bb7df","Image":"php:fpm","Labels":"com.docker.compose.depends_on=,com.docker.compose.image=sha256:c6b042c2a60f05833e6b5041cb5fbd83ef70f42f9b510decb364dc7306dd8da2,com.docker.compose.project.config_files=/var/www/test_docker/docker-compose.yml,com.docker.compose.config-hash=816ffa2d0d86f46676ce2aba77f5d4ded1e538e99c78aee30db77ba4deb5835e,com.docker.compose.container-number=1,com.docker.compose.oneoff=False,com.docker.compose.project=test_docker,com.docker.compose.project.working_dir=/var/www/test_docker,com.docker.compose.service=php,com.docker.compose.version=2.21.0","LocalVolumes":"0","Mounts":"","Name":"test_docker-php-1","Names":"test_docker-php-1","Networks":"test_docker_default","Ports":"9000/tcp","Publishers":[{"URL":"","TargetPort":9000,"PublishedPort":0,"Protocol":"tcp"}],"RunningFor":"6 minutes ago","Service":"php","Size":"0B","State":"running","Status":"Up 6 minutes"}
It is formatted as {...} {...}, but I would rather expect something like [{...}, {...}].
I tested the same on other machines and it works perfectly OK there, so I assume it must be something with my installation / version / configuration.
Some details:
$ docker version
Client: Docker Engine - Community
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:31:44 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.24
GitCommit: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc:
Version: 1.1.9
GitCommit: v1.1.9-0-gccaecfc
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$ docker compose version
Docker Compose version v2.21.0
$ cat ~/.docker/config.json
{
"auths": {
[redacted]
}
}
Below is the output for the same docker-compose.yml file, ran on another server. It has a correct format, but in the same time has a few differences (e.g. no Labels or RunningFor properties, Created vs. CreatedAt, etc.)
$ docker compose ps --format json
[{"ID":"e51334d871cba0ea4cdcd5003b9293b9de7ae01cef0cf95f1df3949f8762066f","Name":"docker_test_nginx_1","Image":"nginx","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"docker_test","Service":"nginx","Created":1695218753,"State":"running","Status":"Up 5 hours","Health":"","ExitCode":0,"Publishers":[{"URL":"","TargetPort":80,"PublishedPort":0,"Protocol":"tcp"}]},{"ID":"438c39a82b6c102eb1b4b0923696b8330a7a2fbcacacb645e7dff69589ea61be","Name":"docker_test_php_1","Image":"php:fpm","Command":"docker-php-entrypoint php-fpm","Project":"docker_test","Service":"php","Created":1695218749,"State":"running","Status":"Up 5 hours","Health":"","ExitCode":0,"Publishers":[{"URL":"","TargetPort":9000,"PublishedPort":0,"Protocol":"tcp"}]}]
Is there anything I should check to make it to produce a properly formatted JSON? Any help will be appreciated!
Unfortunately, this is an expected behavior and breaking change introduced in docker compose 2.21. We are currently dealing with the same issue..
See https://github.com/docker/compose/issues/10958 for more info.
In the same vein from boppy's answer, here's one possible solution that solves both docker compose ps --format json output (no matter which docker compose's version basically):
BEFORE:
docker compose ps -a --format json | jq <...any additional transfo, if any...>
AFTER (the fix):
docker compose ps -a --format json | jq -sc '.[] | if type=="array" then .[] else . end' | jq -s | jq <...any additional transfo, if any...>
For sure, this adds more processing, but in reality, I haven't seen too much impact so far, this will depends on the number of records that you are dealing with.