I'm trying to build a function which I can use as a handler for an RxPy stream that I'm mapping over. The function I have needs access to a variable outside the scope where that variable is defined which, to me, means that I need to use a closure of some kind. So I reached for functools.partial to close over the one variable and return a partial function that I can pass to as an observer to my stream.
However, doing so results in the following:
Traceback (most recent call last):
File "retry/example.py", line 46, in <module>
response_stream = message_stream.flat_map(functools.partial(message_handler, context=context))
File "/home/justin/virtualenv/retry/local/lib/python2.7/site-packages/rx/linq/observable/selectmany.py", line 67, in select_many
selector = adapt_call(selector)
File "/home/justin/virtualenv/retry/local/lib/python2.7/site-packages/rx/internal/utils.py", line 37, in adapt_call_1
argnames, varargs, kwargs = getargspec(func)[:3]
File "/usr/lib/python2.7/inspect.py", line 816, in getargspec
raise TypeError('{!r} is not a Python function'.format(func))
TypeError: <method-wrapper '__call__' of functools.partial object at 0x2ce6cb0> is not a Python function
Here is some sample code which reproduces the problem:
from __future__ import absolute_import
from rx import Observable, Observer
from pykafka import KafkaClient
from pykafka.common import OffsetType
import logging
import requests
import functools
logger = logging.basicConfig()
def puts(thing):
print thing
def message_stream(consumer):
def thing(observer):
for message in consumer:
observer.on_next(message)
return Observable.create(thing)
def message_handler(message, context=None):
def req():
return requests.get('http://httpbin.org/get')
return Observable.start(req)
def handle_response(message, response, context=None):
consumer = context['consumer']
producer = context['producer']
t = 'even' if message % 2 == 0 else 'odd'
return str(message) + ': ' + str(response) + ' - ' + t + ' | ' + str(consumer) + ' | ' + producer
consumer = ['pretend', 'these', 'are', 'kafka', 'messages']
producer = 'some producer'
context = {
'consumer': consumer,
'producer': producer
}
message_stream = message_stream(consumer)
response_stream = message_stream.flat_map(functools.partial(message_handler, context=context))
message_response_stream = message_stream.zip(response_stream, functools.partial(handle_response, context=context))
message_stream.subscribe(puts)
The problem seems to be that my partial function returns False when calling inspect.isfunction.
How can I make my partial function pass this check? Is there a way to easily convert a partial function into a "real" function type?
You're asking if it's actually a function, and it's telling you isn't not a function. It's a method-wrapper.
You want to duck-type.
>>> def printargs(*args):
... print args
>>> import inspect
>>> from functools import partial
>>> inspect.isfunction(printargs)
True
>>> f = partial(printargs, 1)
>>> inspect.isfunction(f)
False
# try duck-typing, see if the variable is callable
# check does it work for a method-wrapper?
>>> callable(f)
True
# check an integer, which should be false
>>> callable(1)
False
# ensure it works on an actual function
>>> callable(printargs)
True
This is why you duck-type. You don't care if it's a function. You care if it acts like a function.
EDIT: You could, if desperate enough, write a class and pass a reference to a function in the class.
class A():
def __init__(self, frozen, *args, **kwds):
self.frozen = frozen
self.args = args
self.kwds = kwds
def call(self):
self.frozen(*self.args, **self.kwds)
Then just use A(f).call as your wrapper.
>>> f_ = A(f)
>>> inspect.ismethod(f_.call)
True
>>> f_.call()
(1,)
This works as long as ismethod works.
If not, you really need a decorator.
Final EDIT: If you are truly desperate enough and don't want to write a custom decorator, you could use a lambda function with a tuple to pass to make a partial-like function.
Ex.:
>>> import inspect
>>> def printargs(*args):
... print args
>>> a = (1,2,3)
>>> f = lambda x: printargs(*x)
>>> f(a)
(1, 2, 3)
>>> inspect.isfunction(f)
True