I have a set of python objects that I create using eulxml.xmlmap.XmlObject (I use this method primarily because I'm working with an eXistDB server and eulxml offers a pretty easy mapping function). I am able to successfully query my eXistDB and load an xquery result set into some python objects I've created. My problem is that I then want to be able to write these objects out as JSON when I pass them to the webserver (using Angular for the front end).
I've tried using jsonpickle but it seems as though eulxml is doing some kind of lazy loading magic. A standard call to jsonpickle to serialize my object to json gives me this result:
python code:
jsonpickle.encode(myObject)
result:
"py/object": "models.alcalaPage.AlcalaPage", "context":
{"namespaces":
{"exist": "http://exist.sourceforge.net/NS/exist"}
},
"node": {
"py/object": "lxml.etree._Element",
"py/seq": [
{"py/object": "lxml.etree._Comment", "py/seq": []},
{"py/object": "lxml.etree._Element", "py/seq": []},
...
]
}...
it seems to be only outputting the type of the attribute but not the value of the attribute itself. If I change my jsonpickle code to set unpickable=False, all I get is an empty set of json (meaning that the structure is there in terms of the right number of curly braces and brackets but there is literally no data. The json output is just curly braces and brackets).
I thought perhaps if I attempted to access a value in the field and then output the json that might work (at least for the field I accessed) but no luck. I get the same result as stated above (and yes I've double checked that there is data in the object itself).
I'm sort of at a loss at this point. I could migrate to something like BeautifulSoup but it means writing A LOT more code (eulxml lets me simply specify the xpath to the value I want to fill my attribute with and bing, I'm done). Is there something I'm missing with jsonpickle? Or is there another json package I should look at? Or maybe I'm making this way more difficult than I need to and there is some other way to query an eXistDB using python and then send the information to a front end application built using Angular. I'm open to suggestions.
I'll include samples of my code below (I won't include all of it because there are probably 10+ objects I'm working with):
Sample object code with eulxml:
import jsonpickle
from eulxml.xmlmap import XmlObject
class AlcalaBase(XmlObject):
def to_xml(self):
return self.serializeDocument(pretty=True)
def to_json(self):
return jsonpickle.encode(self)
from eulxml import xmlmap
from models.alcalaBase import AlcalaBase
class AlcalaPage(AlcalaBase):
ROOT_NAME = 'page'
id = xmlmap.StringField('pageID')
archive_page_number = xmlmap.StringField('archivistsPageNumber')
year = xmlmap.IntegerField('content/@yearID')
I was able to figure out the issue (sort of). So I'm posting it here in case others have the same issue.
The problem seems to be that the attributes are not added to dict so the actual values are not being output during the json process. I wrote my to_json() method in my base class in order to output the appropriate objects. NOTE: While I tried to keep this as generic as possible, it is somewhat specific to my data structure (in that I know what to expect in given scenarios and since I'm dealing with static data, I don't have to "future proof" the solution. Anyone adopting this code should adapt it to their given scenario.
from eulxml import xmlmap
import inspect
import lxml
import json as JSON
class AlcalaBase(xmlmap.XmlObject):
def to_json(self, skipBegin=False):
json = list()
if not skipBegin:
json.append('{')
json.append(str.format('"{0}": {{', self.ROOT_NAME))
for attr, value in inspect.getmembers(self):
if (attr.find("_") == -1
and attr.find("serialize") == -1
and attr.find("context") == -1
and attr.find("node") == -1
and attr.find("schema") == -1):
if type(value) is xmlmap.fields.NodeList:
if len(value) > 0:
json.append(str.format('"{0}": [', attr))
for v in value:
json.append(v.to_json())
json.append(",")
json = json[:-1]
json.append("]")
else:
json.append(str.format('"{0}": null', attr))
elif (type(value) is xmlmap.fields.StringField
or type(value) is str
or type(value) is lxml.etree._ElementUnicodeResult):
value = JSON.dumps(value)
json.append(str.format('"{0}": {1}', attr, value))
elif (type(value) is xmlmap.fields.IntegerField
or type(value) is int
or type(value) is float):
json.append(str.format('"{0}": {1}', attr, value))
elif value is None:
json.append(str.format('"{0}": null', attr))
elif type(value) is list:
if len(value) > 0:
json.append(str.format('"{0}": [', attr))
for x in value:
json.append(x)
json.append(",")
json = json[:-1]
json.append("]")
else:
json.append(str.format('"{0}": null', attr))
else:
json.append(value.to_json(skipBegin=True))
json.append(",")
json = json[:-1]
if not skipBegin:
json.append('}')
json.append('}')
return ''.join(json)
Anything that inherits from this class will be able to serialise out to json. This also assumes that all object collections inherit from this base class (in my particular model, this is true so it's not an issue).