Search code examples
pythonpostgresqlexceptionsqlalchemypyramid

How to catch OperationalError that happened anywhere in Pyramid+SQLAlchemy?


I have a typical Pyramid+SQLAlchemy+Postgres app. In stress testing or during moments of exceptional load and with low max_connections setting in PG it might happen that OperationalException is raised:

OperationalError: (psycopg2.OperationalError) FATAL:  sorry, too many clients already

Now, obviously I do not want to do this everywhere:

try:
    DBSession.query(Item)...
except OperationalError as e:
    log.error(...)

Is there some way of catching this exception "globally" to be properly handled?

My app uses ZopeTransactionExtension in typical Pyramid manner:

DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))

Solution

  • I managed to develop a tween that can do this (example):

    def catch_pg_exc_tween_factory(handler, registry):
        def catch_pg_exc_tween_clos(request):
            response = None
            try:
                response = handler(request)
            except Exception as e:
                log.error('\n\n\n +++ problem: %s', e)
            return response
        return catch_pg_exc_tween_clos
    

    The strange thing is that nothing but explicit tween ordering in development.ini works (no amount of over= or under= tuning of config.add_tween call seems to have worked):

    pyramid.tweens = pyramid_debugtoolbar.toolbar_tween_factory
                     pyramid.tweens.excview_tween_factory
                     pyramid_tm.tm_tween_factory
                     mypkg.util.tweens.catch_pg_exc_tween_factory