I have an equation that can be described by a tree. So the leaves are values with parent vertex being a math operator and when the computation is done, another value appears in the place of parent vertex and it becomes a leaf with a parent vertex(as math operator). The tree is not balanced and can have uneven depth depending on which branch you are pursuing.
What I need to know is how to use Apache ignite (I chose it because of its event propagation mechanism that suits my needs) in order to parallelize by sending all the leaves at different depths(along with their parent vertices) for computation and so on until the equation is calculated.
Am I using wrong technology for this?
Is this problem not suited for map-reduce technologies?
What would be the best technology, which also has event propagation technology built in?
I'm just 1 point short of putting a bounty...
EDIT: Added bounty. Essentially my requirement is computing variable sub problems in a distributed computing solution than similar ones being map-reduced. Any solutions exist? Any thoughts are appreciated.
It's possible with Ignite. You need to implement ComputeTaskAdapter's map() and reduce() methods. First method will map your tasks to nodes, and second collects computation results. Refer example.
After that you may run it again if previous reduce returned collection of results.
To decrease amount of data transferring you may use cache, and store intermediate data in it. If you map tasks to nodes, that have required data, all entries will be available for that tasks locally, and there will not be need to query them from remote node. To map tasks in this manner you may use
ClusterNode dataNode = ignite.affinity("cacheName").mapKeyToNode("key");
Here dataNode is a primary node for this key, and if task will be started on it, the whole entry will be available locally.