Search code examples
dockerdocker-build

How to use buildKit's Go client


I'm trying to prototype a simple image builder using builtKit, Docker's new image builder. I'm not using the command line interface but instead using the client. There's not much documentation on how to use this thing besides the examples.

I need to run a build from a Dockerfile but the example build-using-dockerfile isn't very intuitive. Most importantly, when I run the example with build-using-dockerfile -t 'test:latest' . ('.' being the current directory which contains a simple Dockerfile) the program stalls on

[+] Building 0.0s (0/0)

And is stuck there. How can I do this build using a Dockerfile?


Solution

  • I guess you just followed these steps to configure and build build-using-dockerfile, you also need to configure and install buildkitd daemon.

    This is what I did and it worked.

    • Clone the buildkit project git clone https://github.com/moby/buildkit
    • Configure and install buildkitd and buildctl as mentioned here.
    • Make sure you start buildkitd daemon in background. buildkitd --debug --root /var/lib/buildkit &
    • Now configure and install build-using-dockerfile as mentioned here.
    • Then I build my custom docker image, here is the output I got
    $ build-using-dockerfile -t testimage:v1 .
    DEBU[0020] session started
    [+] Building 0.0s (0/2)
     => [internal] load build definition from Dockerfile                                                                0.0s
    DEBU[0020] new ref for local: x5y2556yo6yml2dzxpe1x1w4f
    DEBU[0020] new ref for local: onaapearn65fka47xr3hu8b8j
    DEBU[0020] diffcopy took: 6.13713ms
    DEBU[0020] diffcopy took: 10.434052ms
    [+] Building 0.1s (1/2)
     => [internal] load build definition from Dockerfile                                                                0.1s
     => => transferring dockerfile: 104B                                                                                0.0s
     => [internal] load .dockerignore                                                                                   0.1s
     => => transferring context: 2B                                                                                     0.0s
    [+] Building 0.7s (2/3)
     => [internal] load build definition from Dockerfile                                                                0.2sn => => transferring dockerfile: 104B                                                                                0.0s, => [internal] load .dockerignore                                                                                   0.1sg => => transferring context: 2B                                                                                     0.0s
     => [internal] load metadata for docker.io/library/nginx:alpine                                                     0.5s
    DEBU[0021] fetch response received                       response.headers="map[Content-Length:[156] Content-Type:[applica[+] Building 1.0s (2/3) => [internal] load build definition from Dockerfile                                                                0.2sk => => transferring dockerfile: 104B                                                                                0.0sy => [internal] load .dockerignore                                                                                   0.1s
    [+] Building 1.2s (2/3) => [internal] load build definition from Dockerfile                                                                0.2s
     => => transferring dockerfile: 104B                                                                                0.0sn => [internal] load .dockerignore                                                                                   0.1s, => => transferring context: 2B                                                                                     0.0sg => [internal] load metadata for docker.io/library/nginx:alpine                                                     0.9s
    DEBU[0021] fetch response received                       response.headers="map[Content-Length:[1412] Content-Type:[application/vnd.docker.distribution.manifest.list.v2+json] Date:[Sat, 22 Jun 2019 11:30:03 GMT] Docker-Content-Digest:[sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63] Docker-Distribution-Api-Version:[registry/2.0] Etag:[\"s[+] Building 1.3s (2/3) => [internal] load build definition from Dockerfile                                                                0.2s
     => => transferring dockerfile: 104B                                                                                0.0s1 => [internal] load .dockerignore                                                                                   0.1s
     => => transferring context: 2B                                                                                     0.0sf => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.1s
    DEBU[0021] do request                                    base="https://registry-1.docker.io/v2/library/nginx" digest="sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63" mediatype=application/vnd.docker.distribution.manifest.list.v2+json request.headers="map[Accept:[application/vnd.docker.distribution.manifest.list.v2+json, *]]" request.method=GET size=1412 url="https://registry-1.docker.io/v2/library/nginx/manifests/sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63"
    DEBU[0021] fetch response received                       base="https://registry-1.docker.io/v2/library/nginx" digest="sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63" mediatype=application/vnd.docker.distribution.manifest.list.v2+json response.headers="map[Content-Length:[1412] Content-Type:[application/vnd.docker.distribution.manifest.list.v2+json] Date:[Sat, 22 Jun 2019 11:30:03 GMT] Docker-Content-Digest:[sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63] Docker-Distribution-Api-Version:[registry/2.0] Etag:[\"sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63\"] Strict-Transport-Security:[max-age=31536000]]" size=1412 status="200 OK" url="https://registry-1.docker.io/v2/library/nginx/manifests/sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63"
    DEBU[0021] fetch                                         digest="sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd8[+] Building 1.5s (2/3)
     => [internal] load build definition from Dockerfile                                                                0.2sa => => transferring dockerfile: 104B                                                                                0.0sf => [internal] load .dockerignore                                                                                   0.1si => => transferring context: 2B                                                                                     0.0sd => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.2s
    DEBU[0022] fetch response received                       base="https://registry-1.docker.io/v2/library/nginx" digest="sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932" mediatype=application/vnd.docker.distribution.manifest.v2+json response.headers="map[Content-Length:[739] Content-Type:[application/vnd.docker.distribution.manifest.v2+json] Date:[Sat, 22 Jun 2019 11:30:03 GMT] Docker-Content-Digest:[sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932] Docker-Distribution-Api-Version:[registry/2.0] Etag:[\"sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932\"] Strict-Transport-Security:[max-age=31536000]]" size=739 status="200 OK" url="https://registry-1.docker.io/v2/library/nginx/manifests/sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932"
    DEBU[0022] fetch                                         digest="sha256:bfba26ca350c153700592ebd7e4613f25a7a8779289e041eb[+] Building 1.8s (2/3)
     => [internal] load build definition from Dockerfile                                                                0.2sa => => transferring dockerfile: 104B                                                                                0.0s1 => [internal] load .dockerignore                                                                                   0.1s= => => transferring context: 2B                                                                                     0.0sd => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.5s
    DEBU[0022] fetch response received                       base="https://registry-1.docker.io/v2/library/nginx" digest="sha256:bfba26ca350c153700592ebd7e4613f25a7a8779289e041eb68a97623277de82" mediatype=application/vnd.docker.container.image.v1+json response.headers="map[Accept-Ranges:[bytes] Cache-Control:[public, max-age=14400] Cf-Cache-Status:[HIT] Cf-Ray:[4eade3578f77cee8-IAD] Content-Length:[7237] Content-Type:[application/octet-stream] Date:[Sat, 22 Jun 2019 11:30:04 GMT] Etag:[\"ef3e5e49b6bd7977fffb3b8cc3729f78\"] Expect-Ct:[max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"] Expires:[Sat, 22 Jun 2019 15:30:04 GMT] Last-Modified:[Tue, 04 Jun 2019 22:32:16 GMT] Server:[cloudflare] Set-Cookie:[__cfduid=da0b42294258d98a7563847f958db178e1561203004; expires=Sun, 21-Jun-20 11:30:04 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; Secure] Vary:[Accept-Encoding] X-Amz-Id-2:[i5ODtFhOarpZHIPlInFHKf2LOdaN43logCkjX0SNk87mY4+pPI36eQ3H0CnWy1lhG05m4NhDKnk=] X-Amz-Request-Id:[A84C182FF892E492] X-Amz-Version-Id:[L2RUincMo02vR2Dt1LK5B1oppj0kjvZ3]]" size=7237 status="200 OK" url="https://registry-1.docker.io/v2/library/nginx/blobs/sha256:bfba26ca350c153700592ebd7e4613f25a7a8779289e041eb68a97623277de82"
    DEBU[0022] fetch                                         digest="sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=1412
    [+] Building 1.9s (3/6)
     => [internal] load build definition from Dockerfile                                                                0.2s8 => => transferring dockerfile: 104B                                                                                0.0s
     => [internal] load .dockerignore                                                                                   0.1s
     => => transferring context: 2B                                                                                     0.0sb => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.6s
     => [1/2] FROM docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.0s
     => => resolve docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.0s
     => [internal] load build context                                                                                   0.0s
     => => transferring context: 73B                                                                                    0.0s
    DEBU[0022] resolving
    DEBU[0022] do request                                    request.headers="map[Accept:[application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, *] Authorization:[Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsIng1YyI6WyJNSUlDK2pDQ0FwK2dBd0lCQWdJQkFEQUtCZ2dxaGtqT1BRUURBakJHTVVRd1FnWURWUVFERXpzeVYwNVpPbFZMUzFJNlJFMUVVanBTU1U5Rk9reEhOa0U2UTFWWVZEcE5SbFZNT2tZelNFVTZOVkF5VlRwTFNqTkdPa05CTmxrNlNrbEVVVEFlRncweE9UQXhNVEl3TURJeU5EVmFGdzB5TURBeE1USXdNREl5TkRWYU1FWXhSREJDQmdOVkJBTVRPMUpMTkZNNlMwRkxVVHBEV0RWRk9rRTJSMVE2VTBwTVR6cFFNbEpMT2tOWlZVUTZTMEpEU0RwWFNVeE1Pa3hUU2xrNldscFFVVHBaVWxsRU1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBcjY2bXkveXpHN21VUzF3eFQ3dFplS2pqRzcvNnBwZFNMY3JCcko5VytwcndzMGtIUDVwUHRkMUpkcFdEWU1OZWdqQXhpUWtRUUNvd25IUnN2ODVUalBUdE5wUkdKVTRkeHJkeXBvWGc4TVhYUEUzL2lRbHhPS2VNU0prNlRKbG5wNGFtWVBHQlhuQXRoQzJtTlR5ak1zdFh2ZmNWN3VFYWpRcnlOVUcyUVdXQ1k1Ujl0a2k5ZG54Z3dCSEF6bG8wTzJCczFmcm5JbmJxaCtic3ZSZ1FxU3BrMWhxYnhSU3AyRlNrL2tBL1gyeUFxZzJQSUJxWFFMaTVQQ3krWERYZElJczV6VG9ZbWJUK0pmbnZaMzRLcG5mSkpNalpIRW4xUVJtQldOZXJZcVdtNVhkQVhUMUJrQU9aditMNFVwSTk3NFZFZ2ppY1JINVdBeWV4b1BFclRRSURBUUFCbzRHeU1JR3ZNQTRHQTFVZER3RUIvd1FFQXdJSGdEQVBCZ05WSFNVRUNEQUdCZ1JWSFNVQU1FUUdBMVVkRGdROUJEdFNTelJUT2t0QlMxRTZRMWcxUlRwQk5rZFVPbE5LVEU4NlVESlNTenBEV1ZWRU9rdENRMGc2VjBsTVREcE1VMHBaT2xwYVVGRTZXVkpaUkRCR0JnTlZIU01FUHpBOWdEc3lWMDVaT2xWTFMxSTZSRTFFVWpwU1NVOUZPa3hITmtFNlExVllWRHBOUmxWTU9rWXpTRVU2TlZBeVZUcExTak5HT2tOQk5sazZTa2xFVVRBS0JnZ3Foa2pPUFFRREFnTkpBREJHQWlFQXFOSXEwMFdZTmM5Z2tDZGdSUzRSWUhtNTRZcDBTa05Rd2lyMm5hSWtGd3dDSVFEMjlYdUl5TmpTa1cvWmpQaFlWWFB6QW9TNFVkRXNvUUhyUVZHMDd1N3ZsUT09Il19.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImxpYnJhcnkvbmdpbngiLCJhY3Rpb25zIjpbInB1bGwiXX1dLCJhdWQiOiJyZWdpc3RyeS5kb2NrZXIuaW8iLCJleHAiOjE1NjEyMDMzMDMsImlhdCI6MTU2MTIwMzAwMywiaXNzIjoiYXV0aC5kb2NrZXIuaW8iLCJqdGkiOiJ6MnlHUk1CdG1RS0g1OXlnUmJXZiIsIm5iZiI6MTU2MTIwMjcwMywic3ViIjoiIn0.dsQw07ln9xAaYqr-qAVv2cgNTzUOKB0W6r8pq3Tc5UF7bl0pEsCIIST-MXpNhVzNuz37Y1RbYh_aa4cKP4Rbxxm00ETOF3sDqxRuodSMIp16L3T2ICjKuftvZCWuDIllgWWJMYeVngoD3tRGx59jUNbs8N02WZ5x-l6DIKpYMfh0IEGZFspePai0a3NjLKlkNqNNnIIO24Pbi1U5F0VAZqRmfkcYyLVFIzji-LEuTI_s8wBRSnAKaXg38WhFicO4td_yVKX4WM2oHDgCPkrzWO-Hf6sDXebMu_agaYKy_ZdARSREU98uO8JN_hJSR3wCRw_8gJTtDrWN0TKd9Noz6w] User-Agent:[containerd/1.2.0+unknown]]" request.method=HEAD url="https://registry-1.docker.io/v2/library/nginx/manifests/sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63"
    DEBU[0022] saved mq410rblsg97q4en3n0rv5mrw as local.sharedKey:context:context:
    DEBU[0022] fetch response received                       response.headers="map[Content-Length:[1412] Content-Type:[application/vnd.docker.distribution.manifest.list.v2+json] Date:[Sat, 22 Jun 2019 11:30:04 GMT] Docker-Content-Digest:[sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63] Docker-Distribution-Api-Version:[registry/2.0] Etag:[\"sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63\"] Strict-Transport-Security:[max-age=31536000]]"status="200 OK" url="https://registry-1.docker.io/v2/library/nginx/manifests/sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63"
    DEBU[0022] resolved                                      desc.digest="sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63"
    DEBU[0022] fetch                                         digest="sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=1412
    DEBU[0022] fetch                                         digest="sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932" mediatype=application/vnd.docker.distribution.manifest.v2+json size=739
    DEBU[0022] fetch                                         digest="sha256:f0e40e45c95e8b730fc083cbfbae0043a261ca769814d32092e89d4d5f959a28" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip size=5684641
    DEBU[0022] fetch                                         digest="sha256:bfba26ca350c153700592ebd7e4613f25a7a8779289e041eb68a97623277de82" mediatype=application/vnd.docker.container.image.v1+json size=7237
    DEBU[0022] fetch                                         digest="sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f10" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip size=2757034
    [+] Building 2.0s (4/6) => [internal] load build definition from Dockerfile                                                                0.2s. => => transferring dockerfile: 104B                                                                                0.0s8 => [internal] load .dockerignore                                                                                   0.1s8 => => transferring context: 2B                                                                                     0.0s
     => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.6sa => [1/2] FROM docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.1s. => => resolve docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.0s5 => [internal] load build context                                                                                   0.0s1 => => transferring context: 73B                                                                                    0.0s
    DEBU[0022] fetch response received                       base="https://registry-1.docker.io/v2/library/nginx" digest="sha[+] Building 2.2s (4/6) => [internal] load build definition from Dockerfile                                                                0.2s[ => => transferring dockerfile: 104B                                                                                0.0sM => [internal] load .dockerignore                                                                                   0.1s/ => => transferring context: 2B                                                                                     0.0s: => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.6s= => [1/2] FROM docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.3sy => => resolve docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.0sR => => sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63 1.41kB / 1.41kB                      0.0s4 => => sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932 739B / 739B                          0.0s
     => => sha256:f0e40e45c95e8b730fc083cbfbae0043a261ca769814d32092e89d4d5f959a28 1.68MB / 5.68MB                      0.2s
    [+] Building 2.3s (4/6)
    [+] Building 2.8s (4/6)
     => [internal] load build definition from Dockerfile                                                                0.2s
    [+] Building 5.4s (4/6)
     => [internal] load build definition from Dockerfile                                                                0.2sa[+] Building 6.7s (6/7)[+] Building 8.5s (6/7)[+] Building 8.5s (7/7) FINISHED => [internal] load build definition from Dockerfile                                                                0.2s/ => => transferring dockerfile: 104B                                                                                0.0s: => [internal] load .dockerignore                                                                                   0.1s= => => transferring context: 2B                                                                                     0.0sA => [internal] load metadata for docker.io/library/nginx:alpine                                                     1.6sP => [1/2] FROM docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  3.7s9 => => resolve docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4  0.0s
     => => sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2b4c63 1.41kB / 1.41kB                      0.0s
     => => sha256:881169baf03885268b54eb07c673bc27f394b263cb728dfd86ff2b65b3450932 739B / 739B                          0.0s
     => => sha256:f0e40e45c95e8b730fc083cbfbae0043a261ca769814d32092e89d4d5f959a28 5.68MB / 5.68MB                      0.3s
     => => sha256:bfba26ca350c153700592ebd7e4613f25a7a8779289e041eb68a97623277de82 7.24kB / 7.24kB                      0.0s6 => => sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f10 2.76MB / 2.76MB                      0.3s
     => => sha256:f0e40e45c95e8b730fc083cbfbae0043a261ca769814d32092e89d4d5f959a28 5.68MB / 5.68MB                      0.3s1 => => unpacking docker.io/library/nginx:alpine@sha256:b126fee6820be927b1e04ae36b3f51aa47d9b73bf6b1826ff19a59d22b2  3.2s
     => [internal] load build context                                                                                   0.0s
     => => transferring context: 73B                                                                                    0.0s
     => [2/2] COPY index.html /usr/share/nginx/html/index.html                                                          1.1s
     => exporting to oci image format                                                                                   1.8s
     => => exporting layers                                                                                             0.9s
     => => exporting manifest sha256:9bb8e9d1941d16ab3ea13fed9b95dff4032ba07e865c6c2d155a8d00cbd9e5a8                   0.0s
     => => exporting config sha256:9acf925d1b3ea1cc98c2053b1c450d993e1ff32dc3514cc90971c21458f25fce                     0.0s
     => => sending tarball                                                                                              0.9s
    f1b5933fe4b5: Loading layer [>                                                  ]  32.77kB/2.757MB
    DEBU[0029] remove snapshot                               key=paqys4xyo4hh9p90ck3u4duxm snapshotter=native
    DEBU[0029] remove content                                key="sha256:9acf925d1b3ea1cc98c2053b1c450d993e1ff32dc3514cc90971c21458f25fce"
    DEBU[0029] remove content                                key="sha256:9bb8e9d1941d16ab3ea13fed9b95dff4032ba07e865c6c2d155a8d00cbd9e5a8"
    DEBU[0029] schedule snapshotter cleanup                  snapshotter=native
    DEBU[0029] schedule content cleanup
    DEBU[0029] removed content                               digest="sha256:9acf925d1b3ea1cc98c2053b1c450d993e1ff32dc3514cc90971c21458f25fce"
    DEBU[0029] removed content                               digest="sha256:9bb8e9d1941d16ab3ea13fed9b95dff4032ba07e865c6c2d155a8d00cbd9e5a8"
    DEBU[0029] content garbage collected                     d=5.429727ms
    DEBU[0029] removed snapshot                              key=buildkit/11/paqys4xyo4hh9p90ck3u4duxm snapshotter=native
    f1b5933fe4b5: Loading layer [==================================================>]  2.757MB/2.757MB
    402522b96a27: Loading layer [==================================================>]  5.685MB/5.685MB
    202e27ce64c9: Loading layer [==================================================>]     299B/299B
    Loaded image: testimage:v1
    INFO[0014] Loaded the image "testimage:v1" to Docker.
    $
    
    • The image was successfully created.
    $ docker images
    REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
    testimage             v1                  9acf925d1b3e        13 seconds ago      20.5MB
    $
    

    In-short build-using-dockerfile uses buildkitd while building the image, which I guess was missing in your case.

    Hope this helps, let me know.