Want to put binary data of images into RocksDB in C++

Question

1. I'm trying to save binary data of images in Key-Value Store
2. 1st, read data using "fread" function. 2nd, save it into RocksDB. 3rd, Get the data from RocksDB and restore the data into form of image.
3. Now I don't know whether I have problem in 2nd step of 3rd step.

2nd step Put

#include <iostream>
#include <string.h> 
#include "rocksdb/db.h"

DB* db;
Options options;
options.create_if_missing = true;


Status s = DB::Open(options, <DBPath>, &db);
assert(s.ok());

//read image
FILE* file_in;
int fopen_err = fopen_s(&file_in, <input_file_path>, "rb");
if (fopen_err != 0) {
    printf(input_file_path, "%s is not valid");;
}

fseek(file_in, 0, SEEK_END);
long int file_size = ftell(file_in);
rewind(file_in);

//malloc buffer
char* buffer = (char*)malloc(file_size);
if (buffer == NULL) { printf("Memory Error!!"); }
fread(buffer, file_size, 1, file_in);

//main func
db->Put(WriteOptions(), file_key, buffer);
assert(s.ok());

fclose(file_in);
free(buffer);
buffer = NULL;
delete db;  

3rd step Get

#include <iostream>
#include <string.h> 
#include "rocksdb/db.h"

DB* db;
Options options;
options.create_if_missing = true;


Status s = DB::Open(options, <DBPath>, &db);
assert(s.ok());

//main func
std::string file_data
s = db->Get(ReadOptions(), file_key, &file_data);
assert(s.ok());

//convert std::string to char*
char* buffer = (char*)malloc(file_data.size() + 1);
std::copy(file_data.begin(), file_data.end(), buffer);

//restore image
FILE* test;
fopen_s(&test, "test.jpg", "wb");
fwrite(buffer, file_data.size(), 1, test);

fclose(test);
free(buffer);
delete db;

The output image is not valid, and if I convert jpg to txt, I only get "???".

I tried on BerkeleyDB in the same process, and I succeed to restore image.(I think it's because of Dbt class of BerkeleyDB) I don't know where the data get crashed. Did I missed some options or process...?

Answer 1

char* buffer = ...
db->Put(WriteOptions(), file_key, buffer);

How is RocksDB supposed to know the length of the buffer? When passing in a char* here, it is assumed to be a nul-terminated C string using the Slice(char *) implicit conversion. Nul-terminated C strings cannot be used for binary data because the data will be cut off at the first zero byte.

Although some RocksDB APIs are not up to modern C++ standards (for API compatibility), it is written for use with C++. Nul-terminated char *, FILE, fseek etc. are from C and cause lots of difficulty when attempting to interact with C++. If buffer were std::string, this bug would be fixed because the Slice(std::string) implicit conversion is very safe.

Other bugs:

• Failure to re-assign s for the db->Put
• Failure to abort on error cases with printf
• Better to call DB::Close(db) before delete to check status, as there could be a background error
• Not checking for error in fread

Performance/clarity issue:

• In 3rd step, no need to create char *buffer and copy in std::string file_data to it. file_data.data() and file_data.size() give you access to the underlying char buffer if needed (but using C++ APIs is better).