Can't use collections.defaultdict() in google-app-engine

Question

Trying to use collections.defaultdict() to create an histogram in google-app-engine :

class myDS(ndb.Model):

    values = ndb.PickleProperty()
    hist = ndb.PickleProperty()

class Handler:
    my_ds = myDS()
    my_ds.values = {}   
    my_ds.hist = defaultdict(lambda : 0) 

And got the error (from log)

File "/base/alloc/tmpfs/dynamic_runtimes/python27/277b61042b697c7a_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1331, in call
    newvalue = method(self, value)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27/277b61042b697c7a_unzipped/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 1862, in _to_base_type
    return pickle.dumps(value, pickle.HIGHEST_PROTOCOL)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Any way to solve this?

Answer 1

A PickleProperty field require a value that is serializable using Python's pickle protocol (see docs for more info):

PickleProperty: Value is a Python object (such as a list or a dict or a string) that is serializable using Python's pickle protocol; Cloud Datastore stores the pickle serialization as a blob. Unindexed by default. Optional keyword argument: compressed.

See also this answer from Martijn Pieters:

Pickle cannot handle lambdas; pickle only ever handles data, not code, and lambdas contain code. Functions can be pickled, but just like class definitions only if the function can be imported. A function defined at the module level can be imported. Pickle just stores a string in that case, the full 'path' of the function to be imported and referenced when unpickling again.

There are multiple options to work with default values, depending on your use case.