Search code examples
c#.netignite

java.OutOfMemoryException when using continuous query in Ignite


I'm using Ignite.NET 2.8.0.20190421 and got OutOfMemoryException when I'm attaching continuous query using ICache.QueryContinuous and listening for an object's changes for a long time.

For example: on the server's side I create the single object in the cache. Then I start the client and start listening for this object's changes using QueryContinuous. And after I start changing the object on the server's side very often and look for its JVM memory consumption in Java VisualVM - it growths constantly until OutOfMemoryException.

It looks like the server holds all the versions of the changing objects in its memory, although they are useless.

Example:

// Our object to store, make it big to easily reproduce the problem
public sealed class DataItem
{
  public int[] Data { get; private set; } = new int[1000000];
}

// Do on the server's side:
// Start Ignite and create cache
while(true)
{
  DataItem item = new DataItem();
  cache.Put(0, item);
  Thread.Sleep(300);
}

// Do on the client's side:
// Start Ignite in client's mode, request cache and start listening
cache.QueryContinuous(query);

Full Visual Studio project, that reproduces the problem:

https://www.dropbox.com/s/aiu3pq0bidkbbif/IgniteListeners.rar?dl=0

Steps:

  1. Start IgniteListeners.exe with any argument to start the server's side, for example, "IgniteListeners.exe s"
  2. Start IgniteListeners.exe with no arguments to start the client
  3. The server will crash within a minute, you can see in Java VisualVM its JVM memory is constantly growing.

Solution

  • It seems to be a known issue. It's related to excessive use of memory in continuous queries, Ignite node needs to store up to 1000 updates in the buffer. It could lead to OOM if entries are heavy. Here you can track progress https://issues.apache.org/jira/browse/IGNITE-11970.