Search code examples
pythonjsonnetwork-programmingremoteobjectpyro

Pyro4 Encoding in JSON/Base64

So I have a simple application using Pyro4 (Python Remote Objects). There is an exposed class API, and I have a file that calls the function api.parse(input,userid), which returns some JSON dependent on input. However, instead of simply returning the result as a string (which it did previously), it returns {'data': 'eyJlcnJvciI6ICJDb21tYW5kIG5vdCByZWNvZ25pc2VkIn0=', 'encoding': 'base64'} , where the base64 is the JSON string that parse should returned.

I'm very confused as to why this is not working - I have tested it previously and there was no issue, the string was just returned with no weird base64 encoding. Only thing I can think of is I have changed networks (School connection to home connection) but I don't think this should be a problem? I have prepared an MVE of some code that indicates the problem.

testserver.py

import Pyro4;
import json;

@Pyro4.expose
class API:
    def parse(self,buff,userid):
        return prep({"error":"Command not recognised"});

def prep(obj):
    return json.dumps(obj).encode("utf-8");

# Pyro stuff
daemon = Pyro4.Daemon()                # make a Pyro daemon
ns = Pyro4.locateNS()                  # find the name server
uri = daemon.register(API)   # register the greeting maker as a Pyro object
ns.register("testAPI", uri)   # register the object with a name in the name server
daemon.requestLoop()

testclient.py

import Pyro4;
import json;

api = Pyro4.Proxy("PYRONAME:testAPI");
resp = api.parse("whatever","something");
print(resp); # Produces {'data': 'eyJlcnJvciI6ICJDb21tYW5kIG5vdCByZWNvZ25pc2VkIn0=', 'encoding': 'base64'}
# I just want b'{"error":"Command not recognised"}'

Note - Printing at the stage where prep is applied in parse() gives the expected result b'{"error":"Command not recognised"}'. I'm using the command python3 -m Pyro4.naming to start the nameserver if that matters as well. I'm thinking there's probably some global setting/constant I haven't set correctly or something - All responses welcome, thankyou!

Solution

  • The default serialization protocol that Pyro uses is serpent, and that is a text-based protocol. This means it cannot transmit binary data (bytes) unless it encodes them to a text format, which it does in the way you discovered.

    There's a little helper function (serpent.tobytes) available that you could use in your client code that automatically detects and converts the response if needed, see the info box in this paragraph in the manual: https://pythonhosted.org/Pyro4/tipstricks.html?highlight=base64#binary-data-transfer-file-transfer

    Ofcourse, if you make the data sent by your server strings in the first place, there's no need for this. (This can't be done if the data really is binary though, such as an image or sound clip or whatever)

    In your case with the json text it gets transformed into bytes if you encode it. As you discovered there is no need for this at all, just leave it as a string and you won't run into issues. (which leaves me with the question why you're still doing the encode to utf-8 bytes in your client?)