Unserializable value

Question

I'm currently stuck with a serialization problem. The data seems ok in Nameko but not in Flask because an error occurred. I tried to serialise and unserialize, but it didn't work.

Here is an example of data from a MongoDB database :

{u'building': u'urk', u'idcapteur': 8, u'room': u'tyu', u'idpiece': 1, u'uri': u'urk/tyu/luminosity/yota1', u'idMesure': 458945, u'subId': u'yota1', u'datemesure2': datetime.datetime(2017, 6, 16, 12, 48, 19, 179000), u'datemesure': datetime.datetime(2017, 6, 16, 12, 48, 19, 179000), u'device': u'luminosity', u'_id': ObjectId('7825ahy'), u'data': {u'date': u'2017-06-16T14:48:19.179435', u'payload': {u'subID': u'yota1', u'input': 50, u'value_units': u'lux', u'value': 500, u'unitID': u'inside'}, u'uri': u'urk/tyu/luminosity/yota1'}, u'mesurevaleur': [{u'idlibv': 5, u'valeur': 500.0}]}

Here is my Nameko program :

# -*-coding:utf-8 -*

# neocampus.py

import json
from database import startMongoDbConnection, getRawDataCollection, mongoDbQuery
from nameko.rpc import rpc
#from database.py import startMongoDbConnection, getRawDataCollection, mongoDbQuery

mdbConnection = ''
mdbCollection = ''

# Send a request to the microservices system. It will call other microservices
class MsRequest:
    name="msRequest"

    # TODO
    @rpc
    def msRequest(self, value):
        # Connect to MongoDB database and collection, if not already done
        global mdbConnection, mdbCollection, queryResult
        if mdbConnection == '':
            mdbConnection = startMongoDbConnection()
        if mdbCollection == '':
            mdbCollection = getRawDataCollection(mdbConnection)

        # Check the requested microservice
        someData = GetData()
        jsonData = someData.getData(value)

        # It prints the proper data
        for document in jsonData:
            print(document)

        print("FINISHED !!!")

        # Don't forget : closeMongoDbConnection()
        return jsonData

# Get data depending on sensor type
class GetData:
    name="getData"

    def getData(self, sensorType):
        global mdbCollection
        queryResult = mongoDbQuery(mdbCollection, "device", "$eq", sensorType)
        return queryResult


# From database.py


def mongoDbQuery(someCollection, key, choice, value):
    # Use MongoDB query dependeing on choices in parameters
    docs = someCollection.find({key:{choice:value}}).limit(10)

    #for document in docs:
        #print(document)
    return docs

# Start a MongoDB connection
def startMongoDbConnection():
    connected = connectToMongoDb("mongodb")
    return connected

# Start a MongoDB connection
def startMongoDbConnection():
    connected = connectToMongoDb("mongodb")
    return connected

# Choose the first collection available (for raw data)
def getRawDataCollection(connection):
    currentCollection = chooseCollection(connection, mongollection)
    return currentCollection

And now, my Flask program :

@app.route('/', methods=['GET', 'POST'])
def getData():
    with ClusterRpcProxy(CONFIG) as rpc:
        myData = rpc.msRequest.msRequest("luminosity")
        #for document in myData:
            #print myData
    return flask.jsonify(**myData)

Finally, the error output :

ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/home/oyo/.local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/oyo/.local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/oyo/.local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/oyo/.local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/oyo/.local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/oyo/flask_sandfox/sandfox.py", line 16, in getData
    myData = rpc.msRequest.msRequest("luminosity")
  File "/home/oyo/.local/lib/python2.7/site-packages/nameko/rpc.py", line 369, in __call__
    return reply.result()
  File "/home/oyo/.local/lib/python2.7/site-packages/nameko/rpc.py", line 327, in result
    raise deserialize(error)
RemoteError: UnserializableValueError Unserializable value: `<pymongo.cursor.Cursor object at 0x7f3274612e50>`

With a print, data is correctly displayed in the terminal. But not in Flask, I get an error 500. I thought that with JSON data and Jsonify (found here, in this site), I could display something. But it's not the cas.

Any help, advice or hint would be gladly appreciated. Thank you.

Answer 1

For people who'll have this exact problem, it took me 5 days to find it. I was sure it was in JSON, because, when I copied/pasted the result, it was ok. In fact, it was in BSON (I didn't even know this format). So, just add this at the beginning of your file :

from bson.json_util import dumps

Then, add this :

# limit for the example
retrievedocs = someCollection.find({* something *}}, {'_id': 0}).limit(10) 
docs = dumps(retrievedocs)

Then, the serialization between Nameko and Flask will work.