Incorrect padding on a Celery SQS consumer

Question

I'm trying to set up a Celery consumer for a SQS queue.

I'm using Celery 4.1.0 with Python3 and I sent a message with the library boto3 (1.5.28) this way:

response = queue.send_message(MessageBody='Hello World')

and everything it's OK so far.

Then I tried to setup a Celery consumer this way:

@app.task(base=celery.Task, name='test', bind=True)
def test(self, message):
    print(message)
    return True

but I got this traceback:

  ...

  File ".../lib/python3.6/site-packages/kombu/transport/SQS.py", line 350, in _on_messages_ready
    msg_parsed = self._message_to_python(msg, qname, queue)
  File ".../lib/python3.6/site-packages/kombu/transport/SQS.py", line 215, in _message_to_python
    body = base64.b64decode(message['Body'].encode())
  File ".../lib/python3.6/base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

Is it anything wrong on what I did? Do I need some specific configuration to work with SQS queues?

Thank in advance!

Answer 1

Your message needs to be a base64 encoded JSON:

message = {"test":"test"}
message_string = json.dumps(message)
byte_message = base64.b64encode(message_string.encode('utf-8'))
base64_json_string = byte_message.decode()
response = queue.send_message(MessageBody=base64_json_string)