Docker Service Update Error: EOF

Question

We have a GO application that listens for code repo changes and Redis messages in order to apply CD in a Docker Swarm environment. That application was recently updated to use the github.com/docker/docker v1.13.1 client code. It worked on my mac, which runs Docker v1.13.1. When I deploy this code to a development server running Ubuntu and Docker v1.12.6, I get an "EOF" error when trying to call cli.ServiceUpdate(..). As far as I know, the CLI client is backwards compatible (until v1.10.x or something). When looking at the code and the screenshot, the connection between the two is the log message of "An error occurred while updating a service".

Event thought we are using the v1.13.1 version of the golang libraries, we specify the api version v1.24 in our client initialization:

func createClient() {
    if cli == nil {
        defaultHeaders := map[string]string{"User-Agent": "engine-api-cli-1.0"}
        var err error
        cli, err = client.NewClient("unix:///var/run/docker.sock", "v1.24", nil, defaultHeaders)
        if err != nil {
            panic(err)
        }
    }
}

Here is the offending code:

func updateService(s configuration.Service, version string) {
    createClient()

    ctx := context.Background()

    service, _, err := cli.ServiceInspectWithRaw(ctx, configuration.ServiceName(s))
    if err != nil {
        fmt.Println("could not find service", err)
        return
    }

    serviceSpec := createServiceSpec(s, version)

    encodedRegistryAuth := createEncodedRegistryAuth()
    var resp types.ServiceUpdateResponse
    resp, err = cli.ServiceUpdate(ctx, service.ID, service.Version, serviceSpec, types.ServiceUpdateOptions{EncodedRegistryAuth: encodedRegistryAuth})
    if err != nil {
        fmt.Println("An error occurred while updating a service: ", s.Service, "; ", err)
    } else {
        message := "SERVICE UPDATE - " + s.Service + "_" + s.Channel + " >> " + envName + ":" + s.Service + ":" + version
        connections.SlackMessage(message)
        connections.NewRelicDeploy(version, message)
        connections.GrafanaDeploy(version, message)
        fmt.Println("Updated service", s.Service, "and waiting", s.UpdateDelay)
        fmt.Println("Warnings from service update", resp.Warnings)
    }
}

Here is a screen grab of what happened in syslog:

Here is the output from docker info:

Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Tue Jan 10 20:40:44 2017
 OS/Arch:      linux/amd64
 Experimental: true

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Tue Jan 10 20:40:44 2017
 OS/Arch:      linux/amd64
 Experimental: true

Any ideas on what I could be doing wrong?

Answer 1

Turned out to be a fluke somewhere. We rebuilt the swarm by stopping the docker daemon, removing /var/lib/docker, and starting the daemon back up. After that the error no longer occurred.