Search code examples
rocksdb

RocksDb does not create SST files for smaller tables


A RocksDB newbie here. I am trying to load data into newly created RocksDB databases. I am using the RocksDB Sharp .NET wrapper around the native Windows RocksDB lib.

Everything seems to be working, but when I peek inside the directories it creates, I only see log files (always 000003.log). I see my data inside, but I thought it is supposed to create something other than the logs. When I try opening it with FastNoSQL, it worked, but, for some reason, I see SST files being created. (Which is probably the normal format.)

I am using WriteBatchWithIndex and I tried setting plain table / block table options. Nothing changes.

I tried tweaking PrepareForBulkLoad, SetAllowMmapWrites and whatnot. The compression is off.

Is that normal?

EDIT: I see that the bigger tables (over 10 Mb) do get SST files. Is there a way to force the smaller ones to "commit" the logs?


Solution

  • OK, figured it myself.

    The easiest way to force RocksDB to create SST immediately was to disable WAL (Write-Ahead Log).

    To the helpful souls that didn't bother answering the question but will want to comment that the approach is wrong: yes, I'm aware of the dangers. In my situation, it's OK. It's a special generation-only project working on the files exclusively, and the results will be distributed to another application.