We've been using RocksDb for years. There's a (.NET) process using RocksDbSharp to generate RocskDb data. There's a C++ library that reads them. Both from 2018 or so. Both use Snappy compression. The C++ library is compiled under C++ 11.
We're trying to upgrade to newer RocksDb now. In unit tests, the new C++ library can create new files and read them just fine. However, it crashes (segmentation fault) on the first rocksdb::DB::Open call when accessing the existing data. More specifically, the call stack goes to LoadTableHandlers.
Here is the source code, nothing fancy:
rocksdb::DB* db;
rocksdb::Options options;
options.compression = rocksdb::CompressionType::kSnappyCompression;
options.create_if_missing = false;
rocksdb::Status status = rocksdb::DB::Open(options, myValidPath, &db);
See the call stack below.
ZwWaitForSingleObject 0x00007ffff97ad144 WaitForSingleObjectEx 0x00007ffff6ef306e pthread_join 0x0000000064945edc _ZNSt6thread4joinEv 0x000000006fce0577 rocksdb::VersionBuilder::Rep::LoadTableHandlers(rocksdb::InternalStats*, int, bool, bool, std::shared_ptr<rocksdb::SliceTransform const> const&, unsigned long long) 0x00000000009747d8 rocksdb::VersionBuilder::LoadTableHandlers(rocksdb::InternalStats*, int, bool, bool, std::shared_ptr<rocksdb::SliceTransform const> const&, unsigned long long) 0x00000000007e86fb rocksdb::VersionEditHandler::LoadTables(rocksdb::ColumnFamilyData*, bool, bool) 0x00000000007ea471 rocksdb::VersionEditHandler::CheckIterationResult(rocksdb::log::Reader const&, rocksdb::Status*) 0x00000000007ec8c8 rocksdb::VersionEditHandlerBase::Iterate(rocksdb::log::Reader&, rocksdb::Status*) 0x00000000007ebe4c rocksdb::VersionSet::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocatorrocksdb::ColumnFamilyDescriptor > const&, bool, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool) 0x00000000005bc135 rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocatorrocksdb::ColumnFamilyDescriptor > const&, bool, bool, bool, unsigned long long, rocksdb::DBImpl::RecoveryContext*) 0x0000000000570caf rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocatorrocksdb::ColumnFamilyDescriptor > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocatorrocksdb::ColumnFamilyHandle* >*, rocksdb::DB**, bool, bool) 0x00000000005698bb rocksdb::DB::Open(rocksdb::Options const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, rocksdb::DB**) 0x000000000056b88f main TestMain.cpp:131 __tmainCRTStartup 0x00000000004013c7 mainCRTStartup 0x00000000004014fb BaseThreadInitThunk 0x00007ffff8a174b4 RtlUserThreadStart 0x00007ffff97626a1 0x0000000000000000
Do we need to tweak something in the options?
Found a workaround. Calling:
options.OptimizeForSmallDb();
before opening eliminates the crash.
Any explanation or analysis from the RocksDb folks?