class instance becomes un(dill)pickleable when constraint function and Parameters defined inside class scope

Question

To diagnose a dill unpickling problem with lmfit objects, I wanted to try including my constraint function definition in the class whose instance I would later pickle. However, dumping a dill pickle where the constraint function is defined inside the class scope raises RuntimeError: maximum recursion depth exceeded (section (3) of code below).

I can't think of an intuitive reason why this should be the case. For example, if I stuff a class-scope function into a class-scope OrderedDict, the class instance remains dill pickleable (section (1) of code below).

Those familiar with lmfit: any idea why lmfit / asteval are triggering this behavior?

Those familiar with OOP but not lmfit: any thoughts on the type of code that might lead to such problems?

Thanks!

import dill
import lmfit
from collections import OrderedDict as od

# Python 2.7.13; dill 0.2.7.1; lmfit 0.9.7; numpy 1.13.3; scipy 1.0.0

#%% 1) this works ####################################
class test(object):

    def inside_func(*args):
        return

    def __init__(self):
        # check OrderedDict dill-pickleable (should be)
        self.d = od([])
        self.d.update({'func': self.inside_func})
        return

t = test()

with open('test.pkl', 'wb') as f:
    dill.dump(t, f)


#%% 2) this also works ###############################

def outside_func(*args):
    return

class test(object):

    def __init__(self):
        # some dummy Parameters set 
        self.p = lmfit.Parameters()
        self.p.add('var1')
        self.p.add('var2')

        # addition of 'func' to _asteval's symtable from outside class scope
        self.p._asteval.symtable['func'] = outside_func
        return

t = test()

with open('test.pkl', 'wb') as f:
    dill.dump(t, f)



#%% 3) this doesn't work ###############################

class test(object):
    def inside_func(*args):
        return

    def __init__(self):
        # some dummy Parmaeters set
        self.p = lmfit.Parameters()
        self.p.add('var1')
        self.p.add('var2')

        # addition of 'func' to _asteval's symtable from inside class scope
        if not any('func' == x for x in self.p._asteval.symtable.keys()):
            self.p._asteval.symtable.update({'func': self.inside_func})
        return

t = test()

with open('test.pkl', 'wb') as f:
    dill.dump(t, f)

Answer 1

You define inside_func at the wrong place. Do it in __init__.

This works:

class test(object):

    def __init__(self):

        def inside_func(*args):
            return

        # some dummy Parmaeters set
        self.p = lmfit.Parameters()
        self.p.add('var1')
        self.p.add('var2')

        # addition of 'func' to _asteval's symtable from inside class scope
        if not any('func' == x for x in self.p._asteval.symtable.keys()):
            self.p._asteval.symtable.update({'func': inside_func})
        return

t = test()

with open('test.pkl', 'wb') as f:
    dill.dump(t, f)

Alternatively, make it a static method:

class test(object):

    @staticmethod
    def inside_func(*args):
        return

    def __init__(self):
        # some dummy Parmaeters set
        self.p = lmfit.Parameters()
        self.p.add('var1')
        self.p.add('var2')

        # addition of 'func' to _asteval's symtable from inside class scope
        if not any('func' == x for x in self.p._asteval.symtable.keys()):
            self.p._asteval.symtable.update({'func': self.inside_func})
        return

t = test()

with open('test.pkl', 'wb') as f:
    dill.dump(t, f)

self.inside_func needs to be a "normal" function. If you do not use staticmethod, Python will bind the method to the instance. This binding triggers the recursion.