Search code examples
luatorch

Grid search for hyper-paramerters in torch / lua

I am new in torch/lua and am trying evaluate some different optimization algorithms and different parameters for each of them.

Algo: optim.sgd optim.lbfgs

Parameters:

  • learning_rate: {1e-1, 1e-2, 1e-3}
  • weight_decay: {1e-1, 1e-2}

So what I am trying to achieve is try every combination of the hyper-parameters and get the optimal parameter set for each of the algorithm.

So is there something like:

param_grid = [
 {'C': [1, 10, 100, 1000], 'kernel': ['linear']},
 {'C': [1, 10, 100, 1000], 'gamma': [0.001, 0.0001], 'kernel': ['rbf']},
]

as in http://scikit-learn.org/stable/modules/grid_search.html available in torch to deal with it?

Any suggestions would be nice!

Solution

  • Try this hyper-optimization library that is being worked on: https://github.com/nicholas-leonard/hypero