Search code examples
pythongoogle-app-enginecompiler-constructionintrospectiondynamic-cast

What are the arguments to the types.CodeType() python call?


I'm currently trying to roll my own "marshal" code for python so i can store compiled python code on Google App Engine to serve scripts on a dynamic way. As you all can verify, "marshal" isn't supported on GAE and "pickle" can't serialize code objects.

I found out i can construct a code object with types.CodeType() but it expects 12 arguments.

As much as i've tried, i can't find any documentation on this call and i really need to construct the code object so i can exec() it. My question is, does anyone know what are the parameters for this types.CodeType() "constructor" or any way to introspect it? i have used the info() function defined here but it spits out just generic info!

Quick FAQ:

  • Q: Why compile the code?
  • A: CPU time costs real money on Google App Engine, and every bit of CPU cycles i can save counts.
  • Q: Why not use "marshal"?
  • A: That's one of the unsupported modules in Google App Engine.
  • Q: Why not use "pickle"?
  • A: Pickle doesn't support serialization of code objects.

UPDATE

Google App Engine infrastructure doesn't allow the instantiation of code objects as of 7th July 2011, so my argument here is moot. Hope this gets fixed in the future on GAE.


Solution

  • I went and took the code found here and removed the dependency for the deprecated "new" module.

    import types, copy_reg
    def code_ctor(*args):
        # delegate to new.code the construction of a new code object
        return types.CodeType(*args)
    def reduce_code(co):
        # a reductor function must return a tuple with two items: first, the
        # constructor function to be called to rebuild the argument object
        # at a future de-serialization time; then, the tuple of arguments
        # that will need to be passed to the constructor function.
        if co.co_freevars or co.co_cellvars:
            raise ValueError, "Sorry, cannot pickle code objects from closures"
        return code_ctor, (co.co_argcount, co.co_nlocals, co.co_stacksize,
            co.co_flags, co.co_code, co.co_consts, co.co_names,
            co.co_varnames, co.co_filename, co.co_name, co.co_firstlineno,
            co.co_lnotab)
    # register the reductor to be used for pickling objects of type 'CodeType'
    copy_reg.pickle(types.CodeType, reduce_code)
    if __name__ == '__main__':
        # example usage of our new ability to pickle code objects
        import cPickle
        # a function (which, inside, has a code object, of course)
        def f(x): print 'Hello,', x
        # serialize the function's code object to a string of bytes
        pickled_code = cPickle.dumps(f.func_code)
        # recover an equal code object from the string of bytes
        recovered_code = cPickle.loads(pickled_code)
        # build a new function around the rebuilt code object
        g = types.FunctionType(recovered_code, globals( ))
        # check what happens when the new function gets called
        g('world')