How to use a boost async_pipe to send child process output across a fork?

Question

I'm new to the boost::asio, and boost::process libraries and I've come across a problem which I'm struggling to find a solution for...

Consider that I have a small toy program that does the following:

• Firstly, fork()s itself into a parent-branch and a child-branch.
• The child-branch then uses the boost::process::child class to invoke the unix command ls in an asynchronous context.
• The child-branch supplies the boost::process::child class with a boost::process::async_pipe to direct std_out to.
• The parent-branch wishes to read what has been written to the pipe, line by line, and process it further.

Currently, my implementation of this works up to a point. However, the read_loop() call in the parent-branch does not terminate. It is almost as if it never reaches EOF, or is blocked. Why is this?

Here is my MWE:

#include <boost/process.hpp>
#include <boost/asio.hpp>
#include <iostream>
#include <string>

#include <unistd.h>

void read_loop(boost::process::async_pipe& pipe)
{
    static boost::asio::streambuf buffer;
    boost::asio::async_read_until(
        pipe,
        buffer,
        '\n',
        [&](boost::system::error_code error_code, std::size_t bytes) {
            if (!error_code) {
                std::istream is(&buffer);
                if (std::string line; std::getline(is, line)) {
                    std::cout << "Read Line: " << line << "\n";
                }
                read_loop(pipe);
            }
            else {
                std::cout << "Error in read_loop()!\n";
                    pipe.close();
            }
        }
    );
}

int main(int argc, char* argv[])
{
    boost::asio::io_context io_context{};
    boost::process::async_pipe pipe{ io_context };

    io_context.notify_fork(boost::asio::io_context::fork_prepare);
    pid_t pid{ fork() };

    if (pid == 0) {
        io_context.notify_fork(boost::asio::io_context::fork_child);
        boost::process::child child(
            boost::process::args({ "/usr/bin/ls", "/etc/" }),
            boost::process::std_out > pipe,
            boost::process::on_exit([&](int exit, std::error_code error_code) { std::cout << "[Exited with code " << exit << " (" << error_code.message() << ")]\n"; }),
            io_context
        );
        io_context.run();
    }
    else {
        io_context.notify_fork(boost::asio::io_context::fork_parent);
        read_loop(pipe);
        io_context.run();
    }

    return 0;
}

Which will successfully give the (abridged) output, as expected:

Read Line: adduser.conf
...
[Exited with code 0 (Success)]
...
Read Line: zsh_command_not_found

but will then just hang until it is forcibly killed.

Which leaves the main question, why does my read_loop() function end up blocking/not exiting correctly?

Thanks in advance!

Answer 1

Chasing The Symptom

The process not "seeing" EOF makes me think you have to close either end of the pipe. This is somewhat hacky, but works:

Live On Coliru

#include <boost/asio.hpp>
#include <boost/process.hpp>
#include <iostream>
namespace bp = boost::process;

void read_loop(bp::async_pipe& pipe) {
    static boost::asio::streambuf buffer;
    using boost::system::error_code;

    async_read_until( //
        pipe, buffer, '\n', [&](error_code ec, [[maybe_unused]] size_t bytes) {
            // std::cout << "Handler " << ec.message() << " bytes:" << bytes << " (" <<
            // buffer.size() << ")" << std::endl;
            if (!ec) {
                std::istream is(&buffer);
                if (std::string line; std::getline(is, line)) {
                    std::cout << "Read Line: " << line << "\n";
                }
                read_loop(pipe);
            } else {
                std::cout << "Loop exit (" << ec.message() << ")" << std::endl;
                pipe.close();
            }
        });
}

int main() {
    boost::asio::io_context ioc{};
    bp::async_pipe          pipe{ioc};

    ioc.notify_fork(boost::asio::io_context::fork_prepare);
    pid_t pid{fork()};

    if (pid == 0) {
        ioc.notify_fork(boost::asio::io_context::fork_child);
        bp::child child( //
            bp::args({"/usr/bin/ls", "/etc/"}), bp::std_out > pipe, bp::std_in.close(),
            bp::on_exit([&](int exit, std::error_code ec) {
                std::cout << "[Exited with code " << exit << " (" << ec.message() << ")]\n";
                pipe.close();
            }),
            ioc);

        ioc.run();
    } else {
        ioc.notify_fork(boost::asio::io_context::fork_parent);
        std::move(pipe).sink().close();
        read_loop(pipe);
        ioc.run();
    }
}

Side note: I guess it would be nice to have a more unhacky way to specify this, like (bp::std_in < pipe).close() or so.

Fixing The Root Cause

When using Boost Process, the fork is completely redundant. Boost Process literally does the fork for you, complete with correct service notification and file descriptor handling.

You'll find the code becomes a lot simpler and also handles the closing correctly (likely because some assumptions within Boost Process implementation details):

Live On Coliru

#include <boost/asio.hpp>
#include <boost/process.hpp>
#include <iostream>
namespace bp = boost::process;

void read_loop(bp::async_pipe& pipe) {
    static boost::asio::streambuf buffer;
    static std::string            line; // re-used because we can
    async_read_until(                   //
        pipe, buffer, '\n',
        [&](boost::system::error_code ec, size_t /*bytes*/) {
            if (ec) {
                std::cout << "Loop exit (" << ec.message() << ")" << std::endl;
                return;
            }
            if (getline(std::istream(&buffer), line))
                std::cout << "Read Line: " << line << "\n";

            read_loop(pipe);
        });
}

int main() {
    boost::asio::io_context ioc{};
    bp::async_pipe          pipe{ioc};

    bp::child child( //
        bp::args({"/bin/ls", "/etc/"}), bp::std_out > pipe,
        bp::on_exit([&](int exit, std::error_code ec) {
            std::cout << "[Exited with " << exit << " (" << ec.message()
                      << ")]\n";
        }));

    read_loop(pipe);
    ioc.run();
}