Boost.Process - reading from process with sleeping loop

Question

I have certain problem and I am not sure what I am doing wrong.

//sleeper.exe

int main()
{
  int i = 0;
  while (true)
  {
    printf("%i\n", ++i);
    sleep_for(1s);
  }

  return 0;
}

I want to capture sleeper's output in my application and add it line by line to some container;

//application.exe

int main()
{
  io_context context;
  async_pipe out(context);
  child sleeper("sleeper.exe", std_out > out, context);      

  vector<string> lines;
  streambuf buffer;
  async_read_until(out, buffer, '\n', [](const error_code& code, size_t size)
  {
    // Add line to container
  });

  context.run();

  return 0;
}

Unfortunately my application hangs on context.run(), probably because sleeper application never terminates. But it should read sleeper's output until delimiter, so I do not know what is the problem here. I am looking forward some explanation.

Edit after more research on topic:

According to: https://support.microsoft.com/en-us/help/190351/how-to-spawn-console-processes-with-redirected-standard-handles

Note Child processes that use such C run-time functions as printf() and fprintf() can behave poorly when redirected. The C run-time functions maintain separate IO buffers. When redirected, these buffers might not be flushed immediately after each IO call. As a result, the output to the redirection pipe of a printf() call or the input from a getch() call is not flushed immediately and delays, sometimes-infinite delays occur. This problem is avoided if the child process flushes the IO buffers after each call to a C run-time IO function. Only the child process can flush its C run-time IO buffers. A process can flush its C run-time IO buffers by calling the fflush() function.

I am still looking for solution in this field.

Answer 1

Indeed you sleeper program doesn't terminate. run() will run till completion.

Let's first make the sample "real" so it has an actual read loop to read more than 1 line:

std::vector<std::string> lines;

boost::asio::streambuf buffer;

std::function<void()> read_loop;
read_loop = [&] {
    boost::asio::async_read_until(out, buffer, "\n", [&](boost::system::error_code code, std::size_t size) {
        if (code) {
            std::cerr << "Oops: " << code.message() << std::endl;
        } else {
            std::cerr << "received: " << size << " bytes" << std::endl;

            auto b = buffers_begin(buffer.data()), m = b+size;
            lines.emplace_back(b, m);

            buffer.consume(size);

            if (lines.size()<10) {
                read_loop();
            }
        }
    });
};

read_loop();

context.run();

You can see that it tries to read 10 lines.

Terminating the child

You can either kill it:

if (lines.size()<10) {
    read_loop();
} else {
    c.terminate();
}

Or close the output pipe, causing the same (broken pipe):

if (lines.size()<10) {
    read_loop();
} else {
    out.close();
}

Deno

I can't make it work on Coliru, but I replaced first.exe with:

#include <iostream>
#include <chrono>
#include <thread>
#include <random>

using namespace std;
static mt19937 prng{random_device{}()};
static auto l() { return uniform_int_distribution(5,20)(prng); }
static auto c() { return uniform_int_distribution('a','z')(prng); }

int main() {
    while(true) {
        cout << std::string(l(), c()) << endl;
        this_thread::sleep_for(chrono::seconds(1));
    }
}

And with the above program, in full:

#include <boost/process.hpp>
#include <boost/asio.hpp>
#include <boost/process/async.hpp>
#include <iostream>
#include <iomanip>
#include <regex>

int main() {
    namespace bp = boost::process;
    using namespace std::string_literals;
    boost::asio::io_context context;
    bp::async_pipe out(context);

    bp::child c("./first.exe", bp::std_out > out, context);

    std::vector<std::string> lines;

    boost::asio::streambuf buffer;

    std::function<void()> read_loop;
    read_loop = [&] {
        boost::asio::async_read_until(out, buffer, "\n", [&](boost::system::error_code code, std::size_t size) {
            if (code) {
                std::cerr << "Oops: " << code.message() << std::endl;
            } else {
                std::cerr << "received: " << size << " bytes" << std::endl;

                auto b = buffers_begin(buffer.data()), m = b+size;
                lines.emplace_back(b, m);

                buffer.consume(size);

                if (lines.size()<10) {
                    read_loop();
                } else {
                    c.terminate();
                }
            }
        });
    };

    read_loop();
    context.run();

    for (auto& line : lines) {
        std::cout << std::quoted(std::regex_replace(line, std::regex("\\n"), "\\n"s)) << "\n";
    }

    return c.exit_code();
}

Printed, on my system:

received: 19 bytes
received: 12 bytes
received: 20 bytes
received: 16 bytes
received: 6 bytes
received: 6 bytes
received: 20 bytes
received: 13 bytes
received: 16 bytes
received: 21 bytes
"dddddddddddddddddd\\n"
"lllllllllll\\n"
"jjjjjjjjjjjjjjjjjjj\\n"
"uuuuuuuuuuuuuuu\\n"
"yyyyy\\n"
"wwwww\\n"
"hhhhhhhhhhhhhhhhhhh\\n"
"qqqqqqqqqqqq\\n"
"aaaaaaaaaaaaaaa\\n"
"xxxxxxxxxxxxxxxxxxxx\\n"