Background task with progress tracking without Celery or Redis

Question

My project do some export from one to another service. For do it, need a long background task and show progress as some text like current step of progress. One problem is how to get this text from long task.

I know about Celery and Redis. But it is needed additional resource like server. The my project is too small and does not count on attendance of more than a couple of people once a month. So I don't want to buy a shared hosting or a machine.

I tried to save this current step text into session. But response during long task is always null. I think because flask is busy with the task and does not return a valid value for the session. Tried to run the task in a new thread. Then I get an error about accessing the session from the wrong thread.

Answer 1

My solution

Create global dict

step = dict()

Create random key and give this key with global step into module a background task. Start a new thread and return key.

@app.route('/api/transfer', methods=['post'])
def api_transfer():
    global step
    key = token_urlsafe(10)

    service = Service(step, key, session.get('token'))

    t = threading.Thread(target=long_task, args=(service,))
    t.start()

    return {'key': key}

And send request with this key to get response current step for it

@app.route('/api/transfer/current_step', methods=['post'])
def api_current_step():
    global step
    return {'step': step[request.json['key']]}

Now for every request to export will create new key and text for only it

{'jlIG7VKtbMHtXg': 'some step for jlIG7VKtbMHtXg'}
{'jlIG7VKtbMHtXg': 'some step for jlIG7VKtbMHtXg', 'wxr4jKyP7c72sg': 'some step for wxr4jKyP7c72sg'}

And at the end of the task, remove this key from the dictionary

del self.step[self.key]