Access violation error while attempting to transfer a large file using HTTP / REST server

Question

Using a REST library, I am trying to set it up as a file sharing server, but am running into issues when transferring large files.

As I understand it, the file transfer should mean opening a stream to the file, getting its buffer in a stringstream,then write it within a response body. This seems to work with small files of only a few bytes or KB, but anything larger fails.

    std::string filePath = "some_accessible_file";
    struct stat st;
    if(stat(filePath.c_str(), &st) != 0)
    {
        //handle it
    }
    size_t fileSize = st.st_size;
    std::streamsize sstreamSize = fileSize;
    std::fstream str; 
    str.open(filePath.c_str(), std::ios::in);
    std::ostringstream sstream;
    sstream << str.rdbuf();
    const std::string str1(sstream.str());
    const char* ptr = str1.c_str();
    response.headers().add(("Content-Type"), ("application/octet-stream"));
    response.headers().add(("Content-Length"), fileSize);
    if (auto resp = request.respond(std::move(response))) //respond returns shared pointer to respond type 
    {
                        
        resp->write(ptr, sstreamSize ); //Access violation for large files
        
    }

Not quite sure why large files would fail. Does file type make a difference? I was able to transfer small text files etc. but a small pdf failed...

Answer 1

The root cause of this error was std::fstream not reading the entire file because it was opened in text mode. In windows, this makes reading stop at a end of file (0x1A) character.

The fix is to open the file in std::ios::binary mode.