Search code examples
azureazure-elastic-scale

Inserting mappings for a large amount of existing data

I am currently testing inserting a large amount of mappings for existing data into a shardmap using Elastic Scale. Turns out the whole process is time consuming. It's inserting around ~10 mappings/second. Is there anyway to speed up the insertion e.g. by inserting batches of mappings or directly via stored procedures?

Solution

  • We know from our own testing that inserting mappings is time consuming. Here are a couple of options I'd suggest you try:

    • You can run multiple parallel threads inserting the mappings.
    • You can increase the service level objective for the shard map database for the time where you do the bulk load.

    I understand why you would want to load mappings in bulk for test scenarios. However, I am not sure I understand the reason why you will need so many mappings that this becomes a problem. Could you explain a bit more?

    Thanks, Torsten