Search code examples
pythonlambdalist-comprehensionkeyword-argument

**kwargs in list comprehension of lambda functions


Problem:

I want to create a list of lambda functions using **kwargs which are iterated in the list.

Similar questions (e.g., e.g.) exist, but they do not consider iterating over **kwargs.

The problem, as before, is that the value of kwargs in the lambda function is evaluated "lazily" after creation in the list, meaning the last value assigned during the iteration is passed to all lambda functions. I've verified this is a 'problem' for classic loops as well as comprehensions.

(2a) is not, having the value argset from the last iteration of the comprehension assigned to all **kwargs. (3a) is worse, having both the values of argset and i from the last iteration assigned to x and **kwargs.

MWE

Code

def strfun(x,**kwargs):
  return 'x: {} | kwargs: {}'.format(x,kwargs)

argsets = [
  {'foo': 'bar'},
  {'baz': 'qux'},
]

# (1) expected behaviour:
print '(1) '+str([strfun(i,**argset) for i,argset in enumerate(argsets)])

# (2) unexpected behaviour:
funs = [lambda x: strfun(x,**argset) for argset in argsets]
print '(2) '+str([fun(i) for i,fun in enumerate(funs)])

# (3) unexpected behaviour:
funs = [lambda : strfun(i,**argset) for i,argset in enumerate(argsets)]
print '(3) '+str([fun() for fun in funs])

Output:

(1) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2) ["x: 0 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(3) ["x: 1 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]

(1) is "correct".

(2) is not, having the last value of argsets assigned to **kwargs for all functions ({'baz': 'qux'}).

(3) is worse, having the last value of both i and argsets assigned to x and **kwargs for all functions (1 and {'baz': 'qux'}).


Solution

  • Solutions

    Solution 1: functools.partial

    As suggested by the solution linked by @Blckknght in the comment above, functools.partial is probably the cleanest way to do this (see below).

    Solution 2: nested lambda

    As suggested by @jfs in this answer, a workaround is to define an outer layer of lambda to force evaluation of the current value of the iterated object during assignment of the inner lambda, and roll over the outer lambda using map, creating the desired list.

    Very minimal code:

    change

    [lambda x: fun(x,**kwargs) for kwargs in kwargset]
    

    to (1)

    [partial(fun, **kwargs) for kwargs in kwargset]
    

    or (2)

    map(lambda kwargs: (lambda x: fun(x,**kwargs)), kwargset)
    

    Full MWE

    Code

    from functools import partial
    
    def strfun(x,**kwargs):
      return 'x: {} | kwargs: {}'.format(x,kwargs)
    
    argsets = [
      {'foo': 'bar'},
      {'baz': 'qux'},
    ]
    # (1) always expected behaviour:
    print '(1)   '+str([strfun(i,**argset) for i,argset in enumerate(argsets)])
    
    # (2)
    # unexpected behaviour:
    funs = [lambda x: strfun(x,**argset) for argset in argsets]
    print '(2-x) '+str([fun(i) for i,fun in enumerate(funs)])
    # expected behaviour
    funs = map(lambda argset: (lambda x: strfun(x,**argset)), argsets)
    print '(2-1) '+str([fun(i) for i,fun in enumerate(funs)])
    # expected behaviour
    funs = [partial(strfun, **argset) for argset in argsets]
    print '(2-2) '+str([fun(i) for i,fun in enumerate(funs)])
    
    # (3)
    # unexpected behaviour:
    funs = [lambda : strfun(i,**argset) for i,argset in enumerate(argsets)]
    print '(3-x) '+str([fun() for fun in funs])
    # expected behaviour
    funs = map(lambda (i,argset): (lambda : strfun(i,**argset)), enumerate(argsets))
    print '(2-1) '+str([fun() for fun in funs])
    # expected behaviour
    funs = [partial(strfun, i, **argset) for i,argset in enumerate(argsets)]
    print '(2-2) '+str([fun() for fun in funs])
    

    Output:

    (1)   ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    (2-x) ["x: 0 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    (2-1) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    (2-2) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    (3-x) ["x: 1 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    (2-1) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    (2-2) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
    

    (2-1),(2-2),(3-1),(3-2) illustrate the possible workarounds.