Prevent Raven from shooting specified exceptions to Sentry

Question

When my database goes down, Sentry instantly gets over flooded by psycopg2's OperationalError: could not connect to server: Connection refused. Since OperationalError can be thrown in other cases than unreachable databases, I can't just blindly ignore it using RAVEN_CONFIG's IGNORE_EXCEPTIONS.

I tried to write a filter for Django logging, but it just doesn't work. It properly intercepts the exception, but still bubbles it up somewhow. Here is the filter:

def skip_unreachable_database(record):
    """Avoid flooding Sentry when the database is down"""
    if record.exc_info:
        print '>>>', record.exc_info
        exc_type, exc_value = record.exc_info[:2]
        if isinstance(exc_value, OperationalError) and exc_value.message.lower().startswith('could not connect to server: connection refused'):
            return False
    return True

There is a ticket about filtering not working with Raven, but it's been closed.

Any idea how I could workaround this?

Answer 1

Here is how I figured it out (for now):

1/ filter out all OperationalError using Raven's configuration:

RAVEN_CONFIG = {
    # [...]
    'IGNORE_EXCEPTIONS': [
        'OperationalError',
    ],
}

2/ add dedicated filter, logger, and logging file for these exceptions, so they don't get lost:

def operational_errors_only(record):
    """Only catch OperationalError exceptions"""
    if record.exc_info:
        exc_type, exc_value = record.exc_info[:2]
        if isinstance(exc_value, OperationalError):
            return True
    return False

LOGGING = {
    # [...]
    'filters': {
        'operational_errors_only': {
            '()': 'django.utils.log.CallbackFilter',
            'callback': operational_errors_only,
        },
    },
    'handlers': {
        'operationalerrors': {
            'mode': 'a',
            'class': 'common_src.logutils.FallbackWatchedFileHandler',
            'filename': '/path/to/operationalerrors.log',
            'formatter': 'verbose',
            'filters': ['operational_errors_only'],
        },
    },
    'loggers': {
        '': {
            'handlers': ['console', 'sentry', 'operationalerrors'],
            'level': 'ERROR',
            'propagate': True,
        },
    },
}