JSON Serialization is empty when serialising from eulxml in python

Question

I am working with eXistDB in python and leveraging the eulxml library to handle mapping from the xml in the database into custom objects. I want to then serialize these objects to json (for another application to consume) but I'm running into issues. jsonpickle doesn't work (it ends up returning all sorts of excess garbage and the value are the fields aren't actually encoded but rather their eulxml type) and the standard json.dumps() is simply giving me empty json (this was after trying to implement the solution detailed here). The problem seems to stem from the fact that the __dict__ values are not initialised __oninit__ (as they are mapped as class properties) so the __dict__ appears empty upon serialization. Here is some sample code:

Serializable Class Object

class Serializable(dict):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # hack to fix _json.so make_encoder serialize properly
        self.__setitem__('dummy', 1)

    def _myattrs(self):
        return [
            (x, self._repr(getattr(self, x)))
            for x in self.__dir__()
            if x not in Serializable().__dir__()
        ]

    def _repr(self, value):
        if isinstance(value, (str, int, float, list, tuple, dict)):
            return value
        else:
            return repr(value)

    def __repr__(self):
        return '<%s.%s object at %s>' % (
            self.__class__.__module__,
            self.__class__.__name__,
            hex(id(self))
        )

    def keys(self):
        return iter([x[0] for x in self._myattrs()])

    def values(self):
        return iter([x[1] for x in self._myattrs()])

    def items(self):
        return iter(self._myattrs())

Base Class

from eulxml import xmlmap
import inspect
import lxml
import json as JSON
from models.serializable import Serializable

class AlcalaBase(xmlmap.XmlObject,Serializable):

    def toJSON(self):
        return JSON.dumps(self, indent=4)

    def to_json(self, skipBegin=False):
        json = list()
        if not skipBegin:
            json.append('{')
        json.append(str.format('"{0}": {{', self.ROOT_NAME))
        for attr, value in inspect.getmembers(self):
            if (attr.find("_") == -1
                and attr.find("serialize") == -1
                and attr.find("context") == -1
                and attr.find("node") == -1
                and attr.find("schema") == -1):
                if type(value) is xmlmap.fields.NodeList:
                    if len(value) > 0:
                        json.append(str.format('"{0}": [', attr))
                        for v in value:
                            json.append(v.to_json())
                            json.append(",")
                        json = json[:-1]
                        json.append("]")
                    else:
                        json.append(str.format('"{0}": null', attr))
                elif (type(value) is xmlmap.fields.StringField
                        or type(value) is str
                        or type(value) is lxml.etree._ElementUnicodeResult):
                        value = JSON.dumps(value)
                        json.append(str.format('"{0}": {1}', attr, value))
                elif (type(value) is xmlmap.fields.IntegerField
                    or type(value) is int
                    or type(value) is float):
                    json.append(str.format('"{0}": {1}', attr, value))
                elif value is None:
                    json.append(str.format('"{0}": null', attr))
                elif type(value) is list:
                    if len(value) > 0:
                        json.append(str.format('"{0}": [', attr))
                        for x in value:
                            json.append(x)
                            json.append(",")
                        json = json[:-1]
                        json.append("]")
                    else:
                        json.append(str.format('"{0}": null', attr))
                else:
                    json.append(value.to_json(skipBegin=True))
                json.append(",")
        json = json[:-1]
        if not skipBegin:
            json.append('}')
        json.append('}')
        return ''.join(json)

Sample Class that implements Base

from eulxml import xmlmap
from models.alcalaMonth import AlcalaMonth
from models.alcalaBase import AlcalaBase

class AlcalaPage(AlcalaBase):
    ROOT_NAME = "page"
    id = xmlmap.StringField('pageID')
    year = xmlmap.IntegerField('content/@yearID')
    months = xmlmap.NodeListField('content/month', AlcalaMonth)

The toJSON() method on the base is the method that is using the Serializable class and is returning empty json, e.g. "{}". The to_json() is my attempt to for a json-like implementation but that has it's own problems (for some reason it skips certain properties / child objects for no reason I can see but thats a thread for another day).

If I attempt to access myobj.keys or myobj.values (both of which are exposed via Serializable) I can see property names and values as I would expect but I have no idea why json.dumps() produces an empty json string.

Does anyone have any idea why I cannot get these objects to serialize to json?! I've been pulling my hair out for weeks with this. Any help would be greatly appreciated.

Answer 1

So after a lot of playing around, I was finally able to fix this with jsonpickle and it took only 3 lines of code:

def toJson(self):
    jsonpickle.set_preferred_backend('simplejson')
    return jsonpickle.encode(self, unpicklable=False)

I used simplejson to eliminate some of the additional object notation that was being added and the unpicklable property removed the rest (I'm not sure if this would work with the default json backend as I didn't test it).

Now when I call toJson() on any object that inherits from this base class, I get very nice json and it works brilliantly.