boost::process environment not being propagated

Question

I am trying to run an external command which uses environment variables to authenticate.

For this I am using boost::process:

namespace bp = boost::process;

std::string exec_bp(const std::string& cmd)
{
    bp::ipstream pipe;
    bp::child c(cmd, bp::std_out > pipe, boost::this_process::environment());

    return std::string(std::istreambuf_iterator<char>(pipe), {});
}

This, however, doesn't work. I get an exception execve failed because the command I'm trying to run cannot find the environment variables it needs.

If I just use popen to run the command and read its stdout (per this answer), it works.

std::string exec_popen(const std::string& cmd)
{
    std::array<char, 128> buffer;
    std::string result;
    std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd.c_str(), "r"), pclose);
    if (!pipe)
        throw std::runtime_error("popen() failed!");
    while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr)
        result += buffer.data();
    return result;
}

As an example, here I am running the aws command line client to list some files in S3:

const std::string cmd = "aws s3 ls s3://foo/bar";

try
{
    auto s = exec_bp(cmd);
    std::cout << "exec_bp:\n" << s << '\n';
}
catch(const std::exception& e)
{
    std::cout << "exec_bp failed; " << e.what() << '\n';
}

try
{
    auto s = exec_popen(cmd);
    std::cout << "exec_popen:\n" << s << '\n';
}
catch(const std::exception& e)
{
    std::cout << "exec_popen failed; " << e.what() << '\n';
}

Output:

$ ./a.out  | head
exec_bp failed; execve failed: Permission denied
exec_popen:
2021-07-05 17:35:08    2875777 foo1.gz
2021-07-05 17:35:09    4799065 foo2.gz
2021-07-05 17:35:10    3981241 foo3.gz

• Why does passing boost::this_process::environment() to boost::process::child not propagate my process's environment?
• How can I use boost::process to execute my command?

Answer 1

To mimic the search-path behaviour of a shell, use boost::process::search_path.

To mimic a shell, use e.g.

bp::child c("/usr/bin/bash",
    std::vector<std::string>{"-c", "aws s3 ls s3://foo/bar"},
    // ...

Note this is typically bad for security. Instead, consider the more direct

bp::child c("/usr/local/bin/aws",
    std::vector<std::string>{"s3", "ls", "s3://foo/bar"},
    // ...