Search code examples
shellsedvariable-substitution

How to substitute values in a key-value file if keys exists in shell environment?

I have a file like below with key value pair separated by =. I have already exported some variables to my shell. Now I want a shell script which can substitute the values for its corresponding keys if those keys are already exist in env variables or exported to shell.

I can have used envsubst but I didn't want to prefix a dollar symbol with those default values. If there is any other tool like envsubst which can help me substitute the string value please refer that also.

foo.yaml

PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network    
ENVIRONMENT=dev
DRUPAL_VERSION=8
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=pass
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
Solution

  • I think, you are looking for something like below.

    > cat foo.yaml
PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network
ENVIRONMENT=dev
DRUPAL_VERSION=8
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=pass
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
>
> export ENVIRONMENT=sit
> export DRUPAL_VERSION=10
> export MYSQL_PASSWORD="*****"
>
> perl -lpe ' s/(.*)=(.*)/sprintf("%s=%s","$1",$ENV{$1}? $ENV{$1}:$2)/ge ' foo.yaml
PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network
ENVIRONMENT=sit
DRUPAL_VERSION=10
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=*****
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
>

    Does this help?.

    Note that you can use -i switch in Perl to overwrite the file if you have write permissions

    > perl -i -lpe ' s/(.*)=(.*)/sprintf("%s=%s","$1",$ENV{$1}? $ENV{$1}:$2)/ge ' foo.yaml
> cat foo.yaml
PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network
ENVIRONMENT=sit
DRUPAL_VERSION=10
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=*****
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
>

    AWK solultion.

    > awk -F"=" ' { $2=ENVIRON[$1]?ENVIRON[$1]:$2; printf("%s=%s\n",$1,$2) } ' foo.yaml

    can be further shortened as 

    > awk -F"=" ' { printf("%s=%s\n",$1,ENVIRON[$1]?ENVIRON[$1]:$2) } ' foo.yaml

    Handling blank lines

    > awk -F"=" ' { if (!/^\s*$/) $0=sprintf("%s=%s",$1,ENVIRON[$1]?ENVIRON[$1]:$2) }1 ' foo.yaml