AWS EB with supervisor running messenger:consume, how to provide env vars safely

Question

I am running a Symfony 4.4 web application on AWS Elastic Beanstalk. I have a very large PDF generating routine I have passed off to Symfony messenger.

Working localy (Ubuntu 18.04) running bin/console messenger:consume works great. It also works if I ssh into an EC2 instance and run it as ec2-user after doing a sudo chmod 777 -R /var/www/html/var/ (otherwise console commads fail unable to write to the var folder)

I have setup the following .ebextesion file that is sucessfully installing supervisor, adding the config files, and starting the process. [It's peiced together from numerous online sources, no I don't REALLY know what I am doing, I think it's a little messy]

files:
  "/tmp/messenger-consume.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      [program:messenger-consume]
      command=php /var/www/html/bin/console messenger:consume async --time-limit=1800 --memory-limit=340M --sleep=30
      user=webapp
      numprocs=1
      autostart=true
      autorestart=true
      process_name=%(program_name)s_%(process_num)02d

  "/tmp/supervisord.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      ; supervisor config file

      [unix_http_server]
      file=/var/run/supervisor.sock   ; (the path to the socket file)
      chmod=0700                       ; sockef file mode (default 0700)

      [supervisord]
      user=root
      logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
      pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
      childlogdir=/var/log/supervisor            ; ('AUTO' child log dir, default $TEMP)

      ; the below section must remain in the config file for RPC
      ; (supervisorctl/web interface) to work, additional interfaces may be
      ; added by defining them in separate rpcinterface: sections
      [rpcinterface:supervisor]
      supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

      [supervisorctl]
      serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket

      ; The [include] section can just contain the "files" setting.  This
      ; setting can list multiple files (separated by whitespace or
      ; newlines).  It can also contain wildcards.  The filenames are
      ; interpreted as relative to this file.  Included files *cannot*
      ; include files themselves.

      [include]
      files = /etc/supervisor/conf.d/*.conf
      ; Change according to your configurations

commands:
  01_01_install_supervisord:
    command: |
      sudo easy_install supervisor
      sudo mkdir -p /etc/supervisor/conf.d/
      sudo mkdir -p /var/log/supervisor
      sudo touch /var/log/supervisor/supervisord.log
    ignoreErrors: true

container_commands:
  01_01_copy_supervisor_configuration_files:
    command: |
      sudo mv /tmp/supervisord.conf /etc/supervisord.conf
      sudo mv /tmp/messenger-consume.conf /etc/supervisor/conf.d/messenger-consume.conf
    ignoreErrors: true
  02_01_start_supervisord:
    command: |
      sudo /usr/local/bin/supervisord -c /etc/supervisord.conf
      sudo /usr/local/bin/supervisorctl reread
      sudo /usr/local/bin/supervisorctl update
      sudo /usr/local/bin/supervisorctl restart all
      sudo /usr/local/bin/supervisorctl start messenger-consume:*
    ignoreErrors: true

I only ever have any success if supervisord is run as root.

If I run my "program" messenger-consume as ec2-user I get the same errors when it tries to write to the web var folder.

if I run messenger-consume as webapp it works fine. So that is the rout I have chosen.

In ALL cases, running messenger-consume as root, ec2-user, or webapp the php enviroment variables come up empty. The variables are requiered for database connection and some other services I am using.

If I populate the .env file with the RDS user, host, password, etc this works. Having security credentials in the .env file is a bad idea, it's in the git repository, and also causes performance issues. [or so I have read, it's just a bad idea]

Is there any way to provide the enviroment variables to Symfony without the .env file? Or how can I have it populated on the fly (maybe via another .ebextension file that will read the php env vars and write the file?) and if I have the .env file and my php env vars, will it still cause performace issues?

I am open to any suggestions on how to make this work. (Symfony 4 messenger + AWS EB)

Answer 1

My solution was to create an .env.prod.local file on deploy by reading the php enviroment variables via getenv()

I put the following code into a symfony console command

        $io = new SymfonyStyle($input, $output);
        $dotenv = new Dotenv(false);

        $env = getenv('APP_ENV');
        $filename = '.env.'.$env.'.local';
        $filePath = $this->root.'/'.$filename;

        if (file_exists($filePath)) {
            $io->error(sprintf('"%s" file exists. It must be removed first!', $filePath));

            return 1;
        }

        $handle = fopen($filePath, 'w+');

        $vars = $dotenv->parse(file_get_contents($this->root.'/.env'));

        foreach ($vars as $key => $value) {
            $envVar = getenv($key) ?: $value;
            $line = $key.'='.$envVar.PHP_EOL;

            fwrite($handle, $line);
        }

        fclose($handle);

        $io->success(sprintf('Generated "%s" file for this "%s" environment', $filename, $env));

        return 0;

and updated the console commands with

. . .
  02_01_start_supervisord:
    command: |
      rm .env.$APP_ENV.local
      php bin/console grid:generate-env-local
      sudo /usr/local/bin/supervisord -c /etc/supervisord.conf
. . .

The idea is to remove an existing .env.prod.local file as it will actually overide any other .env file values, and the regenerate it.