Search code examples
ignite

Behavior of IgniteCache.loadCache

I am using IgniteCache.loadCache(null, keyClassName, sqlArray) to load RDMS data into cache by running sql query specified by sqlArray

It looks that loadCache internally will run the sqlArray with ThreadPool(each sql will be run within a task)

My question is: Does IgniteCache internally will control the parallesm? I have the following scenario:

  1. My datasource's max connection is set to 200.
  2. The sqlArray's length is about 1000 since I have a large table: select * from person where id >=0 and i <=20000 ... select * from person where id >=10000000 and i <=10020000

If all these 1000 sql runs at the same time, then the connection would be unavailable from the connection pool which will lead to error

Solution

  • IgniteCache.loadCache method fully relies on configured CacheStore implementation. It looks like CacheAbstractJdbcStore is support parallelism internally.

    By default, pool size is equal number of available processors, but you are free to change it with CacheAbstractJdbcStore.setMaximumPoolSize(int) method.

    So, you'll run out of connections, if only you have more than 200 processor available.