Stop Sphinx from Executing a cached classmethod property

Question

I am writing a package for interacting with dataset and have code that looks something like

from abc import ABC, ABCMeta, abstractmethod
from functools import cache
from pathlib import Path
from warnings import warn


class DatasetMetaClass(ABCMeta):
    r"""Meta Class for Datasets"""

    @property
    @cache
    def metaclass_property(cls):
        r"""Compute an expensive property (for example: dataset statistics)."""
        warn("Caching metaclass property...")
        return "result"

    # def __dir__(cls):
    #     return list(super().__dir__()) + ['metaclass_property']

class DatasetBaseClass(metaclass=DatasetMetaClass):
    r"""Base Class for datasets that all datasets must subclass"""

    @classmethod
    @property
    @cache
    def baseclass_property(cls):
        r"""Compute an expensive property (for example: dataset statistics)."""
        warn("Caching baseclass property...")
        return "result"


class DatasetExampleClass(DatasetBaseClass, metaclass=DatasetMetaClass):
    r"""Some Dataset Example."""

Now, the problem is that during make html, sphinx actually executes the baseclass_property which is a really expensive operation. (Among other things: checks if dataset exists locally, if not, downloads it, preprocesses it, computes dataset statistics, mows the lawn and takes out the trash.)

I noticed that this does not happen if I make it a MetaClass property, because the meta-class property does not appear in the classes __dir__ call which may or may not be a bug. Manually adding it to __dir__ by uncommenting the two lines causes sphinx to also process the metaclass property.

Questions:

1. Is this a bug in Sphinx? Given that @properties are usually handled fine, it seems unintended that it breaks for @classmethod@property.
2. What is the best option - at the moment - to avoid this problem? Can I somehow tell Sphinx to not parse this function? I hope that there is a possibility to either disable sphinx for a function via comment similarly to # noqa, # type: ignore, # pylint disable= etc. or via some kind of @nodoc decorator.

Answer 1

Everything is working as it should, and there is no "bug" there either in Sphinx, nor in the ABC machinery, and even less in the language.

Sphinx uses th language introspection capabilities to retrieve a class's members and then introspect then for methods. What happens when you combine @classmethod and @property is that, besides it somewhat as a nice surprise actually work, when the class member thus created is accessed by Sphynx, as it must do in search for the doc strings, the code is triggered and runs.

It would actually be less surprising if property and classmethod could not be used in combination actually since both property and classmethod decorators use the descriptor protocol to create a new object with the appropriate methods for the feature they implement. (update: the interaction of property and classmethod was actually complicated enough that this pattern is no longer supported from Python 3.12 onwards)

I think the less surprising thing to go there is to put some explicit guard inside your "classmethod property cache" functions to not run when the file is being processed by sphinx. Since sphinx do not have this feature itself, you can use an environment variable for that, say GENERATING_DOCS. (this does not exist, it can be any name), and then a guard inside your methods like:

...
def baseclass_property(self):
    if os.environ.get("GENERATING_DOCS", False):
        return

And then you either set this variable manually before running the script, or set it inside Sphinx' conf.py file itself.

If you have several such methods, and don't want to write the guard code in all of them, you could do a decorator, and while at that, just use the same decorator to apply the other 3 decorators at once:

from functools import cache, wraps
import os

def cachedclassproperty(func):
     @wraps(func)
     def wrapper(*args, **kwargs):
          if os.environ.get("GENERATING_DOCS", False):
               return
          return func(*args, **kwargs)
     return classmethod(property(cache(wrapper)))

Now, as for using the property on the metaclass: I advise against it. Metaclasses are for when you really need to customize your class creation process, and it is almost by chance that property on a metaclass works as a class property as well. All that happens in this case, as ou have investigated, is that the property will be hidden from a class' dir, and therefore won't be hit by Sphinx introspection - but even if you are using a metaclass for some other purpose, if you simply add a guard as I had suggested might even not prevent sphinx from properly documenting the class property, if it has a docstring. If you hide it from Sphinx, it will obviously go undocumented.