Deserializing YAML back into Python object

Question

I'm trying to write a correct from_yaml method to my class in order to be able to deserialize back to it when loading YAML file using ruamel.yaml library.

Let's assume that in my to_yaml class method I'm returning something like:

@classmethod
def to_yaml(cls, dumper, data):
    dict_representation = {
        'foo': data.foo, 
        'bar': data.bar
    }

    return dumper.represent_mapping(cls.yaml_tag, dict_representation)

Now in the deserialization method

@classmethod
def from_yaml(cls, constructor, node):
    dict_representation = constructor.construct_mapping(node, deep=True)

With this I get the TypeError:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-782b00e19bec> in <module>()
----> 1 dict_representation = yaml.constructor.construct_mapping(node, deep=True)

/home/**/.envs/myenv/local/lib/python2.7/site-packages/ruamel/yaml/constructor.pyc in construct_mapping(self, node, maptyp, deep)
   1186                         "found unhashable key", key_node.start_mark)
   1187             value = self.construct_object(value_node, deep=deep)
-> 1188             self.check_mapping_key(node, key_node, maptyp, key, value)
   1189
   1190             if key_node.comment and len(key_node.comment) > 4 and \

/home/**/.envs/myenv/local/lib/python2.7/site-packages/ruamel/yaml/constructor.pyc in check_mapping_key(self, node, key_node, mapping, key, value)
    241     def check_mapping_key(self, node, key_node, mapping, key, value):
    242         # type: (Any, Any, Any, Any, Any) -> None
--> 243         if key in mapping:
    244             if not self.allow_duplicate_keys:
    245                 args = [

TypeError: argument of type 'NoneType' is not iterable

In fact trying to do this more empirically, in an interactive shell:

import ruamel.yaml
yaml = ruamel.yaml.YAML()
dd = {'foo': 'foo'}
node = yaml.representer.represent_mapping('!dd', dd)
dict_representation = yaml.constructor.construct_mapping(node)

raises the same exception. What am I missing here?

Answer 1

In order for round-tripping to work, the construct_mapping() for the RoundTripConstructor() needs to get the get the actual mapping typ instance passed in, so things like comments can be taken from the node and attached to that instance (normally a CommentedMap()). That extra parameter is not needed when doing a non-roundtrip load (as those don't need to pass on the comment information).

That method could have been designed smarter, as currently if defaults to None as the mapping type if not provided and that is where you get the NoneType is not iterable exception from.

To start with the code at the end of your question, you can just call a simpler mapping constructor by doing:

dict_representation = ruamel.yaml.constructor.SafeConstructor.construct_mapping(
    yaml.constructor, node)

Your from_yaml() class-method should work in the same way:

@classmethod
def from_yaml(cls, constructor, node):
    dict_representation = ruamel.yaml.constructor.SafeConstructor.construct_mapping(
        constructor, node, deep=True)

although you should consider using a two step creation process if you are constructing a complex type (where some indirectly accessible value might reference this node):

@classmethod
def from_yaml(cls, constructor, node):
    dict_representation = dict()
    yield dict_representation
    d = ruamel.yaml.constructor.SafeConstructor.construct_mapping(
         constructor, node, deep=True)
    dict_representation.update(d)