I'm getting ERROR EncodeError(TypeError('6JQAKHNMG9 is not JSON serializable',),)after upgrade.
Successfully installed:
(Initially required for Flower 0.9.1 installation)
Was running:
Now I get:
05/10/2017 06:37:55.045 ERROR EncodeError(TypeError('6JQAKHNMG9 is not JSON serializable',),)
Traceback (most recent call last):
File "/usr/local/src/gonzo/api/version1_0/application/api_main.py", line 191, in post
retries=3)
File "/usr/local/lib/python2.7/dist-packages/celery/app/base.py", line 737, in send_task
amqp.send_task_message(P, name, message, **options)
File "/usr/local/lib/python2.7/dist-packages/celery/app/amqp.py", line 558, in send_task_message
**properties
File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 169, in publish
compression, headers)
File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 252, in _prepare
body) = dumps(body, serializer=serializer)
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 221, in dumps
payload = encoder(data)
File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 54, in _reraise_errors
reraise(wrapper, wrapper(exc), sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 50, in _reraise_errors
yield
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 221, in dumps
payload = encoder(data)
File "/usr/local/lib/python2.7/dist-packages/kombu/utils/json.py", line 72, in dumps
**dict(default_kwargs, **kwargs))
File "/usr/lib/python2.7/json/__init__.py", line 251, in dumps
sort_keys=sort_keys, **kw).encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python2.7/dist-packages/kombu/utils/json.py", line 62, in default
return super(JSONEncoder, self).default(o)
File "/usr/lib/python2.7/json/encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
EncodeError: 6JQAKHNMG9 is not JSON serializable
My code:
Im passing a custom "task_id" which is a string, (not a JSON object) it used to work before the upgrade (If i revert versions it works again), what may be the correct format to pass a custom task_id?
celery.send_task('process_campaign',
exchange='gonzo',
queue='gold',
routing_key='gonzo.gold',
kwargs={'campaign_instance': campaign_instance},
task_id=campaign_instance.reference,
retries=3)
I tried:
task_id=json.dumps(campaign_instance.reference) or
task_id={'task_id': campaign_instance.reference}
Same error, any ideas, I can't find documentation for this issue?
Since the version 4.0, the default serializer in celery is 'json'. Before the default was 'pickle'. (I don't know the details for kombu package versions)
When sending the message you pass a campaign_instance argument which is probably not JSON serializable but is PICKLE serializable that's why it worked before.
You can change the default serializer in the settings specifying 'pickle' which is the shortest way : http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_serializer
Or you can modify your code to pass only JSON serializable arguments to your task.