Hi Stackoverflow community
I have been trying to understand how Django (and Wagtail's Stream-field) work under the hood. Doing that I learned about metaclasses and believe to have a handle on the principle. That said, exactly how SIX executes the with_metaclass function is still a little obscure to me. Here is the code followed by a specific question:
models.py
class BlogPage(Page):
blogElement = StreamField([
('heading', blocks.CharBlock(classname="full title")),
('paragraph', blocks.TextBlock()),
('picture', ImageChooserBlock()),
], default=[])
wagtailcore > fields.py
class StreamField(models.Field):
def __init__(self, block_types, **kwargs):
if isinstance(block_types, Block):
self.stream_block = block_types
elif isinstance(block_types, type):
self.stream_block = block_types()
else:
self.stream_block = StreamBlock(block_types)
super(StreamField, self).__init__(**kwargs)
wagtailcore > blocks > stream_block.py
class StreamBlock(six.with_metaclass(DeclarativeSubBlocksMetaclass, BaseStreamBlock)):
pass
six.py
def with_metaclass(meta, *bases):
"""Create a base class with a metaclass."""
# This requires a bit of explanation: the basic idea is to make a dummy
# metaclass for one level of class instantiation that replaces itself with
# the actual metaclass.
class metaclass(meta):
def __new__(cls, name, this_bases, d):
return meta(name, bases, d)
return type.__new__(metaclass, 'temporary_class', (), {})
QUESTION
(1) The description suggests that we generate a temporary dummy metaclass that replaces itself with the actual metaclass. (2) How does this work? (3) How would we sequence the meta class generation through the with_metaclass function? (4) And where doesBaseStreamBlock come in?
The part that confuses me is is that we define
[1] class metaclass(meta):
but only call it via:
[2] return type.__new__(metaclass, 'temporary_class', (), {})
In [2] we instantiate the class metaclass which we defined in [1]. The instance of this class contains DeclarativeSubBlockMetaclass as the type, and 'temporary_class' as the name with no base or attributes.
In [1] we define the metaclass class which appears to be doing the actual metaclass work. Here we develop a class generator that generates classes of type DeclarativeSubBlockMetaclass (passed in as meta) based on the bases and names.
But, since the only call to [1] comes from [2] all we appear to be doing is to instantiate the 'temporary_class' of type DeclarativeSubBlockMetaclass without any base or attributes.
How do we replace this temporary dummy metaclass with the actual metaclass as described in the description (1)?
I tried to consult the six's docs for this but couldn't find anything that would resolve my confusion.
Any advice would be highly appreciated.
Thank you very much Z
JUST FOR CONTEXT:
I included the code for the two classes used in the six.with_metaclass call above:
DeclarativeSubBlocksMetaclass
class DeclarativeSubBlocksMetaclass(BaseBlock):
"""
Metaclass that collects sub-blocks declared on the base classes.
(cheerfully stolen from https://github.com/django/django/blob/master/django/forms/forms.py)
"""
def __new__(mcs, name, bases, attrs):
# Collect sub-blocks declared on the current class.
# These are available on the class as `declared_blocks`
current_blocks = []
for key, value in list(attrs.items()):
if isinstance(value, Block):
current_blocks.append((key, value))
value.set_name(key)
attrs.pop(key)
current_blocks.sort(key=lambda x: x[1].creation_counter)
attrs['declared_blocks'] = collections.OrderedDict(current_blocks)
new_class = (super(DeclarativeSubBlocksMetaclass, mcs).__new__(mcs, name, bases, attrs))
# Walk through the MRO, collecting all inherited sub-blocks, to make
# the combined `base_blocks`.
base_blocks = collections.OrderedDict()
for base in reversed(new_class.__mro__):
# Collect sub-blocks from base class.
if hasattr(base, 'declared_blocks'):
base_blocks.update(base.declared_blocks)
# Field shadowing.
for attr, value in base.__dict__.items():
if value is None and attr in base_blocks:
base_blocks.pop(attr)
new_class.base_blocks = base_blocks
return new_class
BaseStreamBlock
class BaseStreamBlock(Block):
def __init__(self, local_blocks=None, **kwargs):
self._constructor_kwargs = kwargs
super(BaseStreamBlock, self).__init__(**kwargs)
# create a local (shallow) copy of base_blocks so that it can be supplemented by local_blocks
self.child_blocks = self.base_blocks.copy()
if local_blocks:
for name, block in local_blocks:
block.set_name(name)
self.child_blocks[name] = block
self.dependencies = self.child_blocks.values()
Ok - I think I figured it out. The crux of the matter lies in the
return meta(name, bases, d)
of the with_metaclass function:
def with_metaclass(meta, *bases):
"""Create a base class with a metaclass."""
# This requires a bit of explanation: the basic idea is to make a dummy
# metaclass for one level of class instantiation that replaces itself with
# the actual metaclass.
class metaclass(meta):
def __new__(cls, name, this_bases, d):
return meta(name, bases, d)
return type.__new__(metaclass, 'temporary_class', (), {})
Here is how I think it works in sudo code:
(1) with_metaclass takes <<DeclarativeSubBlocksMetaclass>> as meta; and <<BaseStreamBlock>> as bases
(2) class metaclass(meta) --> the class metaclass is then created extending <<DeclarativeSubBlockMetaclass>> as the class type
(3) def __new__(cls, name, this_bases, d): Only rarely will you have to worry about __new__. Usually, you'll just define __init__ and let the default __new__ pass the constructor arguments to it. __new__ takes care of creating the object and assigning memory space to it. This __new__ method is a class method that gets called when you create an instance of the class and it gets called before __init__. Its main job is to allocate the memory that the object that you are creating uses. It can also be used to set up any aspect of the instance of the class that is immutable Because classes are kind of immutable (they cannot be changed), overloading __new_ is the best place to overload how they are created.
(4) return meta(name, bases, d) --> the class definition ends with returning a <<DeclarativeSubBlockMetaclass>> with the arguments (name, base = BaseStreamBlock, d)
NOTE: We only define the class in 1 - 3; we are not instantiating it this comes below
(5) return type.__new__(metaclass, 'temporary_class', (), {}) --> Here we are using the classic metaclass syntax. This syntax usually looks like this: return type.__new__(cls, name, bases, attrs). We are using this syntax to instantiate the metaclass we defined in (3) and (4). One might think that it is confusing that temporary_class', (), {} are passed on as the 'name', 'bases', and 'attrs' arguments. BUT...
(6) ... when the instantiation arrives at return meta(name,bases,d) we notice that meta doesn't take 'this_bases' as an argument but 'bases'. It derives this value from the arguments which were passed to (1) with_metaclasses. As such bases in this instance == <<BaseStreamBlock>>
(7) Therefore, when we instantiate type.__new__(metaclass, 'temporary_class', (), {}) we essentially execute <<DeclarativeSubBlocksMetaClass>>('temporary_class', <<BaseStreamBlock>>, {})
The step explained in (7) is what the explanation talked about. Essentially what SIX does is to go through the prescribed steps to create a dummy metaclass which it calls temporary_class. Since DeclarativeSubBlocksMetaClass is also a metaclass, it then uses the BaseStreamBlock base to generate a new class.
I hope this makes sense.
Z