Python class scoping rules

Question

EDIT: Looks like this is a very old "bug" or, actually, feature. See, e.g., this mail

I am trying to understand the Python scoping rules. More precisely, I thought that I understand them but then I found this code here:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        print(x)
        print(y)
        y = 1
func()

In Python 3.4 the output is:

xlocal
ytop

If I replace the inner class by a function then it reasonably gives UnboundLocalError. Could you explain me why it behaves this strange way with classes and what is the reason for such choice of scoping rules?

Answer 1

TL;DR: This behaviour has existed since Python 2.1 PEP 227: Nested Scopes, and was known back then. If a name is assigned to within a class body (like y), then it is assumed to be a local/global variable; if it is not assigned to (x), then it also can potentially point to a closure cell. The lexical variables do not show up as local/global names to the class body.

---

On Python 3.4, dis.dis(func) shows the following:

>>> dis.dis(func)
  4           0 LOAD_CONST               1 ('xlocal')
              3 STORE_DEREF              0 (x)

  5           6 LOAD_CONST               2 ('ylocal')
              9 STORE_FAST               0 (y)

  6          12 LOAD_BUILD_CLASS
             13 LOAD_CLOSURE             0 (x)
             16 BUILD_TUPLE              1
             19 LOAD_CONST               3 (<code object C at 0x7f083c9bbf60, file "test.py", line 6>)
             22 LOAD_CONST               4 ('C')
             25 MAKE_CLOSURE             0
             28 LOAD_CONST               4 ('C')
             31 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
             34 STORE_FAST               1 (C)
             37 LOAD_CONST               0 (None)
             40 RETURN_VALUE

The LOAD_BUILD_CLASS loads the builtins.__build_class__ on the stack; this is called with arguments __build_class__(func, name); where func is the class body, and name is 'C'. The class body is the constant #3 for the function func:

>>> dis.dis(func.__code__.co_consts[3])
  6           0 LOAD_NAME                0 (__name__)
              3 STORE_NAME               1 (__module__)
              6 LOAD_CONST               0 ('func.<locals>.C')
              9 STORE_NAME               2 (__qualname__)

  7          12 LOAD_NAME                3 (print)
             15 LOAD_CLASSDEREF          0 (x)
             18 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             21 POP_TOP

  8          22 LOAD_NAME                3 (print)
             25 LOAD_NAME                4 (y)
             28 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             31 POP_TOP

  9          32 LOAD_CONST               1 (1)
             35 STORE_NAME               4 (y)
             38 LOAD_CONST               2 (None)
             41 RETURN_VALUE

Within the class body, x is accessed with LOAD_CLASSDEREF (15) while y is load with LOAD_NAME (25). The LOAD_CLASSDEREF is a Python 3.4+ opcode for loading values from closure cells specifically within class bodies (in previous versions, the generic LOAD_DEREF was used); the LOAD_NAME is for loading values from locals and then globals. However closure cells show up neither as locals nor globals.

Now, because the name y is stored to within the class body (35), it is consistently being used as not a closure cell but a local/global name. The closure cells do not show up as local variables to the class body.

This behaviour has been true ever since implementing PEP 227 - nested scopes. And back then BDFL stated that this should not be fixed - and thus it has been for these 13+ years.

---

The only change since PEP 227 is the addition of nonlocal in Python 3; if one uses it within the class body, the class body can set the values of the cells within the containing scope:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        nonlocal y  # y here now refers to the outer variable
        print(x)
        print(y)
        y = 1

    print(y)
    print(C.y)

func()

The output now is

xlocal
ylocal
1
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    func()
  File "test.py", line 13, in func
    print(C.y)
AttributeError: type object 'C' has no attribute 'y'

That is, print(y) read the value of the cell y of the containing scope, and y = 1 set the value in that cell; in this case, no attribute was created for the class C.