Search code examples
pythondockershellconsul

How do I use the Consul Agent CLI to create new KV entries but only if they don't already exist?


I have a toy Consul in a docker-compose.yml file. It gets populated with another service that runs and exits.

version: '3.4'
services:
  consul:
    image: consul
  consul-seed:
    build:
      context: consul/seed
    environment:
      HELLO: WORLD

The consul/seed contains a Dockerfile that runs a Python script.

FROM python:3-latest
RUN python3 -m pip install python-consul # Actually, this is via a requirements.txt. But for simplicity, this works too.
COPY seed.py ./
CMD python3 seed.py

The python script just reads some environment variables and populates Consul's KV with them.

#!/usr/bin/env python3

import os
import consul

seed_keys=["HELLO"]
consul = consul.Consul()
for key in seed_keys:
  value = os.environ.get(key, "")
  if consul.kv.put(key=key, value=value, cas=0):
    print("{}={}".format(key, value))
  else:
    print("! {}={}".format(key, value))

That works! When I do: consul kv get HELLO, the contents are WORLD. Importantly, the seed script doesn't overwrite any value which already has a key (demonstrating that the cas option is respected and the seed script should be safe to run with a "real" Consul instance instead of the toy one).

$ docker exec -ti consul /bin/sh
/ # consul kv get HELLO
WORLD
/ # consul kv put HELLO wOrLd
Success! Data written to: HELLO
/ # consul kv get HELLO
wOrLd

However if I try to use the Consul CLI to seed a value, it complains about the CAS options.

$ docker exec -ti consul /bin/sh
/ # consul kv get HELLO
wOrLd
/ # consul kv delete HELLO
Success! Deleted key: HELLO
/ # consul kv put -cas HELLO world
Must specify -modify-index with -cas!
/ # consul kv put -cas 0 HELLO world
Error! Too many arguments (expected 1 or 2, got 3)
/ # consul kv put -cas -modify-index 0 HELLO world
Must specify -modify-index with -cas!
/ # consul kv put -cas -modify-index 1 HELLO world
Error! Did not write to HELLO: CAS failed
/ # consul kv get HELLO
Error! No key exists at: HELLO

How do I use the agent CLI to create new KV entries but only if they don't already exist?

I am using:

$ docker pull consul
Using default tag: latest
latest: Pulling from library/consul
Digest: sha256:0e660ca8ae28d864e3eaaed0e273b2f8cd348af207e2b715237e869d7a8b5dcc
Status: Image is up to date for consul:latest
docker.io/library/consul:latest
$ docker image ls consul
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
consul              latest              941109e2896d        4 weeks ago         122MB
$ docker run --rm -ti consul consul --version
Consul v1.8.0
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

Solution

  • I'm a product manager for Consul. This is a bug in our CLI. We filed https://github.com/hashicorp/consul/issues/8330 to track fixing this issue.

    In the mean time you can use the HTTP API directly, or via a SDK, to conditionally set keys if they don't exist.