Customize streambuffer for C++ ostream

Question

I am implementing my own streambuffer for output stream. Basically it is a vector-like streambuffer in which everytime the overflow function simply reallocates the buffer to two times larger. The sync function will write all data out to the device specified by a file descriptor fd.

class MyStreamBuf : public ::std::streambuf {

  constexpr static size_t INIT_BUFFER_SIZE {1024};

  public: 

    MyStreamBuf();
    ~MyStreamBuf();

    void fd(const int);

    int sync() override;
    int_type overflow(int_type ch = traits_type::eof()) override;

  private:

    int _fd {-1};
    size_t _size;
    char_type* _base;    
    void _resize(const size_t);
};


MyStreamBuf::MyStreamBuf() {
  _size = INIT_BUFFER_SIZE;
  _base = static_cast<char_type*>(malloc(_size * sizeof(char_type)));
  setp(_base, _base + _size - 1);   // -1 to make overflow easier.
}

// Destructor.
MyStreamBuf::~MyStreamBuf() {
  ::free(_base);
}

// Procedure: fd
// Change the underlying device.
void MyStreamBuf::fd(const int fd) {
  _fd = fd;
}

// Procedure: _resize
// Resize the underlying buffer to fit at least "tgt_size" items of type char_type.
void MyStreamBuf::_resize(const size_t tgt_size) {

  // Nothing has to be done if the capacity can accommodate the file descriptor.
  if(_size >= tgt_size) return;

  // Adjust the cap to the next highest power of 2 larger than num_fds
  for(_size = (_size ? _size : 1); _size < tgt_size; _size *= 2);

  // Adjust and reset the memory chunk.
  _base = static_cast<char_type*>(::realloc(_base, _size*sizeof(char_type)));

  setp(_base, _base + _size - 1);   // -1 to make overflow easier.
}

int MyStreamBuf::sync() {

  int res = 0;

  ::std::ptrdiff_t remain = pptr() - pbase();

  while(remain) {

    issue_write:
    auto ret = ::write(_fd, pptr() - remain, remain);

    if(ret == -1) {
      if(errno == EINTR) {
        goto issue_write;
      }
      else if(errno == EAGAIN) {
        break;
      }
      else {
        res = -1;
        break;
      }
    }
    remain -= ret;
  }

  if(remain) {
    ::memcpy(pbase(), pptr() - remain, remain*sizeof(char_type));
  }
  pbump(pbase() + remain - pptr());

  return res;
}

typename MyStreamBuf::int_type MyStreamBuf::overflow(int_type ch) {
  assert(traits_type::eq_int_type(ch, traits_type::eof()) == false);
  _resize(_size * 2);
  return ch;
}

However I am getting segfault while replacing the cout with my own buffer. I couldn't find where the error is after struggling with GDB.

// Function: main
int main() {

  auto fd = open("./test.txt",  O_WRONLY | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR);

  MyStreamBuf d;

  d.fd(fd);

  ::std::cout.rdbuf(&d);

  ::std::cout << 1 << " " << 2 << ::std::endl;

  close(fd);

  return 0;
}

Is there anything wrong with this implementation? I saw many articles typically overriding sync and overflow are required.

Answer 1

The problem, it seems, is that your object d is destroyed before std::cout, and thus the final calls for destructing the global object, which include flushing buffers, and that take palce after the end of main() (remember it's a global object), attempt to perform operations on a no longer-extant streambuf object. Your buffer object definitely should outlive the stream you associate it with.

One way of having this in you program is to make d into a pointer, which you will never delete. Alternatively, you can keep your local object as you used it, but call std::cout.flush(), and then assign cout's buffer to something else (even nullptr) before going out of scope.

While testing with your program (and before I found the problem), I made small changes that made sense to me. For example, after you successfully write to the descriptor, you can simply bump(ret) (you already know that ret!=-1, so its safe to use).

Other changes that I didn't make, but which you could consider, are to have the descriptor set by the constructor itself, having the destructor close a dangling descriptor, and perhaps change dynamic allocation from C-oriented malloc()/realloc()/free() to C++-oriented std::vector.

Speaking of allocation, you made a very common mistake when using realloc(). If the reallocation fails, realloc() will keep the original pointer intact, and signal the failure by returning a null pointer. Since you use the same pointer to get the return value, you risk losing the reference to a still allocated memory. So, if you at all cannot use C++ containers instead of C pointers, you should change you code to something more like this:

char *newptr;
newptr=static_cast<char *>(realloc(ptr, newsize));
if(newptr)
    ptr=newptr;
else {
    // Any treatment you want.  I wrote some fatal failure code, but
    // you might even prefer to go on with current buffer.
    perror("ralloc()");
    exit(1);
}