I am fitting GARCH models on data from thousands of sensors, for each day of the month using the ARCH package. I know that all data are not clean, and the model might not converge for some sensors, and I am OK with that. I plan to deal with them on a per sensor basis later.
My problem is the way Python handles Warnings. As per the Warnings documentation:
Conceptually, the warnings filter maintains an ordered list of filter specifications; any specific warning is matched against each filter specification in the list in turn until a match is found; the match determines the disposition of the match.
Which basically means that
warnings.simplefilter('ignore')
will be appended to the head of the list.
However, in the ARCH package, in /arch/base.py, line 507 reads:
warnings.simplefilter('always')
which in essence appends 'always' to the start of the warnings filter each time a call is made to the model fit method of ARCH. This ensures that the warning is always displayed because I can only add 'ignore' to the head of the list either before or after placing a call to .fit() (which would be overridden by 'always' in the next call. Since my problem involves thousands of sensors, it prints thousands of warnings which slow the Jupyter notebook to a crawl.
Is there a way to ignore warnings under all conditions? Like a super filter for warnings would be great.
They're resetting the filter every time, so I don't see any other solutions than hijacking the warn function.
When you import a Python module, it gets stored in the dictionary sys.modules for later uses. Therefore, it is sufficient to import the warnings module once and before the ARCH package:
import warnings
warnings.warn = lambda *a, **kw: False
# do stuff which might trigger warnings
It's an ugly solution, I'll admit. But as a quick hack it should serve its purpose.
As a long term solution, I would advise you to open a PR and explain your situation. Simply adding an argument to the function to decide whether to emit warnings or not seems like a good idea to me.