Is it safe to ignore the "ConvergenceWarning" in sklearn?

Question

I'm using LassoCV in sklearn. There's always a lot of ConvergenceWarnings. I checked the results and they look good. So I was wondering if it is safe to simply ignore those warnings. A few thoughts I have are:

• Does the warning happens because the magnitude of my response is too big? It will make the loss bigger, too.
• In most cases, would it be solved when I increase the number of iterations? I'm hesitating to do it because sometimes it takes longer to run but the results didn't improve.

Answer 1

tl;dr It is fine almost always, to be sure watch learning curve.

So, LassoCV implements Lasso regression, which parameters are optimized via some kind of gradient descent (coordinate descent to be more precise, which is even simpler method) and as all gradient methods this approach requires defining:

1. step size
2. stop criterion

probably most popular stop criteria are:

a) fixed amount of steps (good choice time-wise, since 1000 steps takes exactly x1000 time compare to 1 step, so it is easy to manage the time spent on training).

b) fixed delta (difference) between values of a loss function during step n and n-1 (possibly better classification/regression quality)

The warning you observe is because LassoCV uses the the first criterion (fixed amount of steps), but also checks for the second (delta), once number of fixed steps is reached the algorithm stops, default value of delta is too small for most real datasets.

To be sure that you train your model long enough you could plot a learning curve: loss value after every 10-20-50 steps of the training, once it goes to a plateau you are good to stop.