How is __dir__ implemented exactly, and how should I know it?

Question

So I'm trying to know the details of dir() function. First I looked at the implementation of it: https://github.com/python/cpython/blob/e76daebc0c8afa3981a4c5a8b54537f756e805de/Objects/object.c#L1450-L1477

/* Helper for PyObject_Dir: object introspection. */
static PyObject *
_dir_object(PyObject *obj)
{
    PyObject *result, *sorted;
    PyObject *dirfunc = _PyObject_LookupSpecial(obj, &PyId___dir__);

    assert(obj);
    if (dirfunc == NULL) {
        if (!PyErr_Occurred())
            PyErr_SetString(PyExc_TypeError, "object does not provide __dir__");
        return NULL;
    }
    /* use __dir__ */
    result = _PyObject_CallNoArg(dirfunc);
    Py_DECREF(dirfunc);
    if (result == NULL)
        return NULL;
    /* return sorted(result) */
    sorted = PySequence_List(result);
    Py_DECREF(result);
    if (sorted == NULL)
        return NULL;
    if (PyList_Sort(sorted)) {
        Py_DECREF(sorted);
        return NULL;
    }
    return sorted;
}

And found that the _dir_object function doesn't do any work itself, but invokes the __dir__ method of the introspected object.

>>> def test(): pass
>>> test.__dir__
<built-in method __dir__ of function object at 0x10ee57ae8>

So how to know the implementation of it?

Answer 1

__dir__ is a special method, so looked up on the type, at least in Python 3:

>>> type(test)
<class 'function'>
>>> '__dir__' in dir(type(test))
True
>>> type(test).__dir__
<method '__dir__' of 'object' objects>
>>> dir(test) == sorted(type(test).__dir__(test))
True

See the Special method lookup section of the datamodel:

For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.

This is exactly what the _PyObject_LookupSpecial() function does, see the typeobject.c source code:

res = _PyType_LookupId(Py_TYPE(self), attrid);

The Py_TYPE() call there is the important part, __dir__ is looked up on the type.

The __dir__ method is implemented on the object type and inherited by the function type, so the implementation is in the object_dir() function.

For Python 2, the dir() implementation is more elaborate, and actually also delegates to other functions! For function objects, it delegates to the _generic_dir() function. This function consults the __dict__ of the type:

/* Merge in attrs reachable from its class. */
itsclass = PyObject_GetAttrString(obj, "__class__");
if (itsclass == NULL)
    /* XXX(tomer): Perhaps fall back to obj->ob_type if no
                   __class__ exists? */
    PyErr_Clear();
else {
    if (merge_class_dict(dict, itsclass) != 0)
        goto error;
}

where merge_class_dict() recursively incorporates the class hierarchy attributes into the final result.