I was looking for how celery and sqs deletion works? When does celery delete message from sqs? Does it delete when message is picked from sqs or after completion of tasks? What happens if there is some error in task or it raises error? Will the message (task) will be there if the tasks is taking too long like and 20 mins.
When does celery delete message from sqs? Message will be deleted after completing the task.
What happens if there is some error in task or it raises error? Message is still with broker, and deleted after max_retries reaches.
Will the message (task) will be there if the tasks is taking too long like and 20 mins? This depends on visibility timeout. Message goes to "Not Visible" state, till your visibility timeout, after that it is available to worker. (if visibility timeout is less than retry time, worker will consume same message many times).
Best Practice is (visibility timeout) > (max_retries * retry_time)