Search code examples
singlestorecolumnstore

MemSQL - Columnstore: individual insert improvements in 4.1


The MemSQL 4.1 release notes advise that there have been improvements to insert performance of Columnstore tables.

My (basic) understanding of the Columnstore table type is that it is unsuitable for individual inserts and best placed for larger bulk inserts (~100k rows per insert).

Is this still the case with the 4.1 release or does the memory-optimized data structure in front of each columnstore table now fix this deficiency?

To be clear, performance is not the issue for my use case, its utilizing the colunstore for individual inserts.

Appreciate any additional information or links to further reading - I cannot find more details on these changes.


Solution

  • That's right, with 4.1, MemSQL columnstore tables can support individual inserts reasonably (unlike before 4.1 where each individual insert created a separate columnstore segment). Individual inserts go into in-memory row storage until enough accumulate that we can batch them up into column storage. Of course MemSQL will be able to handle bulk inserts with much better perf.