We are developing a prediction model using deepchem's GCNModel.
Model learning and performance verification proceeded without problems, but it was confirmed that a lot of time was spent on prediction.
We are trying to predict a total of 1 million data, and the parameters used are as follows.
model = GCNModel(n_tasks=1, mode='regression', number_atom_features=32, learning_rate=0.0001, dropout=0.2, batch_size=32, device=device, model_dir=model_path)
I changed the batch size to improve the performance, and it was confirmed that the time was faster when the value was decreased than when the value was increased.
All models had the same GPU memory usage.
From common sense I know, it is estimated that the larger the batch size, the faster it will be. But can you tell me why it works in reverse?
We would be grateful if you could also let us know how we can further improve the prediction time.
let's clarify some definitions first.
Epoch
Times that your model and learning algorithm will walk through your dataset.
(Complete passes)
BatchSize
The number of samples(every single row of your training data) before updating the internal model. in other words, the number of samples processed before the model is updated.
So Your batch size is something between 1 and your len(training_data)
Generally, more batch size gives more accuracy of training data.
Epoch ↑ Batch Size ↑ Accuracy ↑ Speed ↓
So the short answer to question is more batch size takes more memory and needs more process and obviously takes longer time to learn.
https://stats.stackexchange.com/questions/153531/what-is-batch-size-in-neural-network
Here is the link for more details.