How to apply 'load_from' and 'dump_to' to every field in a marshmallow schema?

Question

I've been trying to implement an 'abstract' schema class that will automatically convert values in CamelCase (serialized) to snake_case (deserialized).

class CamelCaseSchema(marshmallow.Schema):
    @marshmallow.pre_load
    def camel_to_snake(self, data):
        return {
            utils.CaseConverter.camel_to_snake(key): value for key, value in data.items()
        }

    @marshmallow.post_dump
    def snake_to_camel(self, data):
        return {
            utils.CaseConverter.snake_to_camel(key): value for key, value in data.items()
        }

While using something like this works nicely, it does not achieve everything applying load_from and dump_to to a field does. Namely, it fails to provide correct field names when there's an issue with deserialization. For instance, I get: {'something_id': [u'Not a valid integer.']} instead of {'somethingId': [u'Not a valid integer.']}.

While I can post-process these emitted errors, this seems like an unnecessary coupling that I wish to avoid if I'm to make the use of schema fully transparent.

Any ideas? I tried tackling the metaclasses involved, but the complexity was a bit overwhelming and everything seemed exceptionally ugly.

Answer 1

You're using marshmallow 2. Marshmallow 3 is now out and I recommend using it. My answer will apply to marshmallow 3.

In marshmallow 3, load_from / dump_to have been replace by a single attribute : data_key.

You'd need to alter data_key in each field when instantiating the schema. This will happen after field instantiation but I don't think it matters.

You want to do that ASAP when the schema is instantiated to avoid inconsistency issues. The right moment to do that would be in the middle of Schema._init_fields, before the data_key attributes are checked for consistency. But duplicating this method would be a pity. Besides, due to the nature of the camel/snake case conversion the consistency checks can be applied before the conversion anyway.

And since _init_fields is private API, I'd recommend doing the modification at the end of __init__.

class CamelCaseSchema(Schema):

    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        for field_name, field in self.fields.items():
            fields.data_key = utils.CaseConverter.snake_to_camel(field_name)

I didn't try that but I think it should work.