Search code examples
consulconsul-template

Combine 2 KV folder in consul with http endpoint

I created 4 kv folder(i mean directory. Each pair contain 10's of key-value pair) on consul for my testing jenkins pipeline. So that i can create .env file for my app. These are public_stable, private_stable, public_temp, private_temp. I want to combine public_stable and public_temp in a new KV folder with new name like public_my_testing_env_name. The same for private values. I want to combine because my coworker will update running environment's values. So that they can find their .env file inside thousands of .env file easily with testing_env_name.

1-) Can i do that with consul http endpoint without reading all of the values? I dont want to do it programmatically.

2-) I read values with consul-template. Can i create that folder with consul-template?

3-) This is not legal i know :| . Do u think this way is a good one?

Solution

  • You can achieve this by using either envconsul or Teller.

    Quoting envconsul's docs, "Envconsul provides a convenient way to launch a subprocess with environment variables populated from HashiCorp Consul and Vault."

    Teller, on the other hand, supports reading data from other providers, in addition to Consul.

    Here are two examples of using both envconsul and Teller to read keys from public_stable/ and public_temp/, and makes those available as environment variables to the launched sub-process.

    First, create a test key under each of these paths.

    $ consul kv put public_stable/STABLE_URL https://example.com/stable_website
Success! Data written to: public_stable/STABLE_URL

$ consul kv put public_temp/TEMP_RELOAD_INTERVAL 600
Success! Data written to: public_temp/TEMP_RELOAD_INTERVAL

    envconsul

    Download envconsul. Set the CONSUL_HTTP_ADDR environment variable to the address of your Consul client's API. If ACLs are enabled in the environment, you will also need to configure the CONSUL_HTTP_TOKEN environment variable.

    export CONSUL_HTTP_ADDR=http://localhost:8500
export CONSUL_HTTP_TOKEN=<token>

    Run the following command which instructs envconsul to query the keys under the public_temp and public_stable, and make them available as environment variables. envconsul will run the env command so that we can see the set of variables that are provided to the sub-process.

    $ envconsul -pristine -prefix public_temp -prefix public_stable -once env
TEMP_RELOAD_INTERVAL=600
STABLE_URL=https://example.com/stable_website

    Teller

    Install Teller and create .teller.yml in your project's directory with the following contents.

    .teller.yml

    ---
opts:
  environment: public

# Providers
providers:
  # Configure via environment:
  # CONSUL_HTTP_ADDR
  # CONSUL_HTTP_TOKEN, if ACLs are enabled
  consul:
    env_sync:
      path: "{{environment}}_stable/"
  consul2:
    kind: consul
    env_sync:
      path: "{{environment}}_temp/"

    Use teller env to output the retrieved key-value pairs in .env file format in order to validate the data is correctly being merged into a single set of environment variables.

    $ teller env
TEMP_RELOAD_INTERVAL=600
STABLE_URL=https://example.com/stable_website

    You can then run your application under Teller using teller run. Details of this command's syntax are documented in https://github.com/SpectralOps/teller#running-running-subprocesses.