Search code examples
pythonsqlalchemygetattr

Using __getattr__ with SQLAlchemy ORM leads to RecursionError


Simple self-contained example below, presupposing SQLite.

I'm using the SQLAlchemy (v1.3) ORM, where I have a table of world-given whatsits that should not be changed. I also have another table with the same whatsits, in a form more usable to the developer. For instance, this table of dev-whatsits has fields to keep cached results of complex calculations made on data in the raw whatsits. The dev-whatsits table is connected to the raw-whatsits table through its ID as a foreign key; this is also modelled as a (one-way) relationship in SQLAlchemy.

This works fine. Now, frequently while interacting with a dev-whatsit, the developer will want to look at attributes in the underlying raw version. This is simple enough:

result = dev_instance.raw_whatsit.some_attribute

However, since it's the same real-world object that is represented, it would be more convenient and intuitive to be able to skip the middle bit and write:

result = dev_instance.some_attribute

I thought this would be reasonably simple using __getattr__, e.g. like this:

    def __getattr__(self, item):
        try:
            getattr(self.raw_whatsit, item)
        except AttributeError as e:
            # possibly notify here?
            raise e

However, this leads to a RecursionError: maximum recursion depth exceeded after going back and forth between the getattr here and the line return self.impl.get(instance_state(instance), dict_) in InstrumentedAttribute.__get__ in sqlalchemy\orm\attributes.py.

Is there a better way of "redirecting" attribute access in the way I want? Or is there a simple fix I have not yet found?

Self-contained code giving RecursionError follows. Comment out AppWhatsit.__getattr__ and the very last print statement to make it work.

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, sessionmaker, scoped_session
from sqlalchemy import create_engine, Column, ForeignKey, Integer, String

Base = declarative_base()

class RawWhatsit(Base):
    '''Data for whatsit objects given by the world, with lots and lots of attributes.
    This table should not be changed in any way.'''
    __tablename__ = 'raw_whatsits'

    whatsit_id = Column(Integer, primary_key=True)
    one_of_many_attributes = Column(Integer)
    another_attribute = Column(String(16))

    def __init__(self, wid, attr, attr2):
        '''Just a helper for this demonstration.'''
        self.whatsit_id, self.one_of_many_attributes, self.another_attribute = wid, attr, attr2

class AppWhatsit(Base):
    '''A model of a whatsit intended to be used by the developer.
    It has a separate db table and, for instance, caches results of lengthy calculations.
    Has foreign key back to corresponding raw whatsit.'''
    __tablename__ = 'app_whatsits'

    whatsit_id = Column(Integer, ForeignKey('raw_whatsits.whatsit_id'), primary_key=True)
    result_of_complex_calc = Column(Integer)

    raw_whatsit = relationship('RawWhatsit')

    def __init__(self, raw_instance):
        self.whatsit_id = raw_instance.whatsit_id

    def do_complex_calc(self):
        self.result_of_complex_calc = (self.raw_whatsit.one_of_many_attributes +
                                       len(self.raw_whatsit.another_attribute))

    # Attempt at making attributes of the raw whatsits more easily accessible. Leads to bottomless recursion.
    # (Comment out this and the very last print statement below, and the code works.)
    def __getattr__(self, item):
        try:
            getattr(self.raw_whatsit, item)
        except AttributeError as e:
            # possibly notify here?
            raise e

def run():
    # Set up database stuff:
    engine = create_engine('sqlite:///:memory:', echo=False)
    Base.metadata.create_all(engine)
    Session = scoped_session(sessionmaker(bind=engine))  # a class
    session = Session()

    # Populate raw table (in reality, this table is given by the world):
    raw_instance = RawWhatsit(1, 223, 'hello')
    session.add(raw_instance)
    session.commit()
    print(session.query(RawWhatsit).first().__dict__) # ... 'whatsit_id': 1, 'one_of_many_attributes': 223, ...

    # Later: Create a developer-friendly whatsit object associated with the raw one:
    raw_instance_from_db = session.query(RawWhatsit).first()
    dev_instance = AppWhatsit(raw_instance_from_db)
    session.add(dev_instance)
    session.commit()
    dev_instance.do_complex_calc()
    print(session.query(AppWhatsit).first().__dict__)   # ... 'result_of_complex_calc': 228, 'whatsit_id': 1, ...

    # All is good. Now I want to see some of the basic data:
    print(dev_instance.raw_whatsit.another_attribute)   # hello
    # ...but I'd prefer to be able to write:
    print(dev_instance.another_attribute)

if __name__ == '__main__':
    run()

Solution

  • SQLAlchemy keeps the session state of an instance in the _sa_instance_state attribute, which is set on an instance if the instance doesn't yet have one. To test if an instance has the attribute, however, it has to call __getattr__ of the instance to query the name _sa_instance_state, so your overridden __getattr__ should raise an AttributeError in this case to allow the initialization logics to instantiate a state object for the instance:

    def __getattr__(self, item):
        if item == '_sa_instance_state':
            raise AttributeError
        return getattr(self.raw_whatsit, item)
    

    Demo: https://replit.com/@blhsing1/AwareMerryPatches