Search code examples
pythonmachine-learningscikit-learnlasso-regression

what is max_iter and tol in Lasso Regularizer sklearn


According to the documentation https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html, what does the max_iter and tol mean? Also how do I decide the value for max_iter and tol in order to have more control on the optimization?


Solution

  • A lasso regression has a unique optimum, but the solver is a sort of gradient descent algorithm, so you'll never actually reach the minimum. tol controls how close you want to be: the smaller tol, the more accurate your final solution will be, but the longer it will take. max_iter controls how many steps you'll take in the gradient descent before giving up. The algorithm will stop when either updates are within tol or you've run for max_iter many steps; if the latter, you'll get a warning saying that the model hasn't converged (to within tol).

    So, set tol to your liking, and set max_iter according to your computational resources. Usually, go with the defaults and increase max_iter (and/or change solver, or scale your data if you haven't yet) if you get convergence warnings.