Search code examples
c++bashboostboost-regexboost-iostreams

how to convert bash script to C++ using boost::iostreams


I'm trying to convert the following bash code into C++ using boost::iostreams:

#!/usr/bin/bash
(
    gzip -cd file1.ext.gz
    cat file2.ext
) | grep '^regex' # or sed 's/search/replace/'

I can open a file and decompress it:

std::ifstream s("file.ext.gz", std::ios_base::in | std::ios_base::binary);
boost::iostreams::filtering_istreambuf in;
in.push(boost::iostreams::gzip_decompressor());
in.push(s);

Then open an uncompressed file:

std::ifstream s2("file.ext", std::ios_base::in | std::ios_base::binary);

Now I'm a bit stuck, so here are my questions:

1) What's the boost::iostreams solution to concat the two streams?

2) How to output the result through a regex filter to emulate grep/sed?

As a result I'd like to have a an istream that i can copy to cout:

boost::iostream::copy(result, std::cout);

UPDATE complete solution using Hamigaki's concatenate:

/*
 * convert the following bash script into C++
 *
 * #!/bin/bash
 * (
 *     gzip -cd file1.ext.gz
 *     cat file2.ext
 * ) | grep '^filter' | 'sed s/search/replace/g'
 *
 */

#include <iostream>
#include <boost/bind.hpp>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/filter/regex.hpp>
#include <boost/iostreams/filter/grep.hpp>
#include <boost/iostreams/copy.hpp>

// http://hamigaki.sourceforge.jp/hamigaki/iostreams/concatenate.hpp
#include "concatenate.hpp"

namespace io = boost::iostreams;

int main(int argc, char const* argv[])
{
    io::file_source file1("file1.ext.gz");
    io::file_source file2("file2.ext");
    io::gzip_decompressor gzip;
    io::regex_filter sed(boost::regex("search"), "replace");
    io::grep_filter grep(boost::regex("^filter"));

    io::filtering_istreambuf in1(gzip | file1);
    io::filtering_istreambuf in2(file2);

    io::filtering_istreambuf combined(sed | grep | 
            hamigaki::iostreams::concatenate(
                boost::ref(in1),
                boost::ref(in2)
            )
        );

    io::copy(combined, std::cout);

    return 0;
}

Solution

  • 1) I don't know if there's anything built into boost, but this class seems to be exactly what you want: http://hamigaki.sourceforge.jp/hamigaki/iostreams/concatenate.hpp

    The catch here is that it expects CopyConstructible devices to concatenate and Chains seem to not be CopyConstructible. However, we can easily work around that using boost::ref. This code does (almost) what I understood you're asking:

    int main(int argc, char const* argv[])
    {
      boost::iostreams::filtering_istreambuf in;
      boost::regex regex("search");
      boost::iostreams::regex_filter rf(regex, "replace");
      in.push(rf);
    
      boost::iostreams::file_source file1(argv[1]);
      in.push(file1);
    
      boost::iostreams::file_source file2(argv[2]);
      boost::iostreams::copy(hamigaki::iostreams::concatenate(boost::ref(in), file2), std::cout);
    
      return 0;
    }
    

    I just used the regex filter instead of gzip, for testing.

    2) boost::iostreams has a regex filter: http://www.boost.org/doc/libs/1_45_0/libs/iostreams/doc/classes/regex_filter.html

    EDIT: You seem to have this working, now.