Ignite cache invoke update very very slow when value is Treemap and data in treemap increase upto 10K

Question

I have code logic of update ignite cache record,

the cache is defined as:

IgniteCache<String, TreeMap<Long, InfoResultRecord>> txInfoCache;

The key is cache type string, and for value i use TreeMap for keep the record in order(I need the data to be ordered), but the time used for update is increasing with the TreeMap size increase, what i found is when the TreeMap size around 10K, each invoke for one record add to the cache value treemap is very slow, around 2 seconds, if i have 1K data need add to the Treemap, it will cost 2000 seconds, it is really very slow and not acceptable.

I use invoke to update the cache to add record to the Treemap:

txInfoCache.invoke(txType, new TxInfoProcessor(), record);

config for the cache is:

CacheConfiguration<String, TreeMap<Long, InfoResultRecord>> cacheCfg =
        new CacheConfiguration<>("TxInfoCache");
cacheCfg.setCacheMode(CacheMode.REPLICATED);
//cacheCfg.setStoreKeepBinary(true);
cacheCfg.setAtomicityMode(ATOMIC);
cacheCfg.setBackups(0);
txInfoCache = ignite.getOrCreateCache(cacheCfg);

and the processor for add record to Treemap is:

private static class TxInfoProcessor implements EntryProcessor<
        String,
        TreeMap<Long, InfoResultRecord>,
        TreeMap<Long, InfoResultRecord>> {

    @Override
    public TreeMap<Long, InfoResultRecord> process(
            MutableEntry<String,
                    TreeMap<Long, InfoResultRecord>> entry, Object... args) {

        InfoResultRecord record = (InfoResultRecord) args[0];

        final Long oneDayMsSeconds = 24 * 60 * 60 * 1000L;

        TreeMap<Long, InfoResultRecord>
                InfoResultRecordTreeMap = entry.getValue();

        if (InfoResultRecordTreeMap == null) {
            InfoResultRecordTreeMap = new TreeMap<>();
        }


        InfoResultRecordTreeMap.put(record.getDealTime() + oneDayMsSeconds, record);
        entry.setValue(InfoResultRecordTreeMap);

        return null;
    }
}

Is there something wrong? or i use the cache in the wrong way?

I also write a simple test code to veriy the speed when get/put with TreeMap:

public class Server2 {

    public static void main(String[] args) throws IgniteException {
        try (Ignite ignite = Ignition.start("server-start.xml")) {

            IgniteCache<String, TreeMap<Long, String>> testCache = ignite.getOrCreateCache("testCache");
            testCache.put("my",new TreeMap<>());

            while (true) {
                StopWatch stopWatch = new StopWatch();
                stopWatch.start("1");
                TreeMap<Long, String> map = testCache.get("my");
                stopWatch.stop();
                stopWatch.start("2");
                map.put(System.currentTimeMillis(),String.valueOf(new Random().nextInt(1000000000)));
                testCache.put("my",map);
                stopWatch.stop();

                System.out.println("cacheSize:"+map.size()+","+stopWatch.prettyPrint());
            }
        }
    }
}


cacheSize:1000,StopWatch '': running time (millis) = 195
-----------------------------------------
ms     %     Task name
-----------------------------------------
00080  041%  1
00115  059%  2

cacheSize:1001,StopWatch '': running time (millis) = 38
-----------------------------------------
ms     %     Task name
-----------------------------------------
00028  074%  1
00010  026%  2


cacheSize:3000,StopWatch '': running time (millis) = 139
-----------------------------------------
ms     %     Task name
-----------------------------------------
00055  040%  1
00084  060%  2

cacheSize:3001,StopWatch '': running time (millis) = 68
-----------------------------------------
ms     %     Task name
-----------------------------------------
00042  062%  1
00026  038%  2

It shows clearly when the Treemap size raise, the time consume of ignite cache get/put increase, i think this should be 1~2ms, but here it's xx ms and with size increase it will to xxxms even seconds.

Answer 1

It's obvious, that in such case read/write time will grow, since on each operation it will need to copy data from off-heap to heap(and vice-versa) and serialize/deserialize it, however, I've checked your code locally and time grew not so drastically. For example, I got

cacheSize:10082,StopWatch '': running time (millis) = 18
-----------------------------------------
ms     %     Task name
-----------------------------------------
00009  050%  1
00009  050%  2

Anyway, I don't think that you chose a good solution for your use case - I'd recommend storing all this data separately, i.e introduce key with fields for TreeMap string name and certain long key for a row and store these data by rows. Using that, your write operations will be much faster. Also, in this case, you will need to collocate data by treemap name. To request all the data you can use SQL with ORDER BY statement. And don't forget to use indexes!