Does h2o make use of all the available cores on an ec2 server (not instance) on the fly?
I ask because I run several instances of rstudio on ec2. Some are bigger, some are smaller. My spaghetti code gets transferred from one window to another.
When setting cores on a t2.micro
h2o.init(nthreads = -1)
It will connect to the "max" number of cores I explicitly created for the instance. However,
H2O cluster total nodes: 1
H2O cluster total memory: 0.23 GB
H2O cluster total cores: 15
H2O cluster allowed cores: 1
Seems to suggest that there are more cores available.
I noticed this when I transferred code over from another instance with 16 cores to the instance with 1 core.
h2o.init(nthreads = 16)
This code created
H2O cluster total nodes: 1
H2O cluster total memory: 0.23 GB
H2O cluster total cores: 15
H2O cluster allowed cores: 15
Another example, on the 16 core instance, I was curious, and set nthreads=128, but it maxes out at 40.
Is h2o grabbing extra allowable cores on the fly? I ask because this would be much easier and quicker than setting up a cluster.
As mentioned above, t2.micro is a single core instance, so the H2O cluster allowed cores: 1 is correct. However, it looks like you are running into bug in reporting the number of H2O cluster allowed cores with virtual machines (like those used in EC2) that was documented here. This was fixed on 4/1/16 and so it should be fixed in the latest stable release. Download the latest stable of H2O and try again -- if you still see H2O cluster total cores: 15, post a comment here and we will re-open the ticket.