Search code examples
pythonmultithreadingpython-3.xuwsgipython-multithreading

Python3 threading with uWSGI


I wasted many time but couldn't find a solution.
If i use threads in my app deployed with uwsgi, they aren't sync.

Here simple code for an example(wsgi.py):

from time import sleep
import threading

i = 0
def daemon():
  global i
  while True:
    i += 1
    print(i)
    sleep(3)
th = threading.Thread(target=daemon, args=())
th.start()

def application(environ, start_response):
  start_response('200 OK', [('Content-Type','text/html')])
  return [str(i).encode()]

And when I run this app the i increases in log, but I always get 1 when a make request from browser.(Or get 0 if I move sleep(3) before i first increment)
I tried uwsgi.thread decorator, but got the same result.

uwsgi config:

[uwsgi]
socket = 127.0.0.1:3034
plugins-dir = /srv/uwsgi
plugin = python34
uid = py3utils
gid = py3utils
chdir = /srv/python/3/py3utils/tht/app/
wsgi-file = wsgi.py
enable-threads = true
daemonize = %(chdir)/../uwsgi.log
master = true
die-on-term = true
touch-reload = ../uwsgi_restart.txt

*sorry for my English


Solution

  • This happens because after importing your application the master process forks into a worker:

    spawned uWSGI master process (pid: 7167)
    spawned uWSGI worker 1 (pid: 7169, cores: 1)
    spawned uWSGI http 1 (pid: 7170)
    

    So your thread which prints i is running in master process, and your requests are processed by the worker. The worker during the fork sees i equal to 1. If you move sleep before incrementing i the process manages to fork before the first increment.

    Threads except the main one are not copied during a fork, so i does not increment in the worker.

    You should use something like uwsgidecorators.thread:

    from time import sleep
    import threading
    import uwsgidecorators
    
    i = 0
    
    @uwsgidecorators.postfork
    @uwsgidecorators.thread
    def daemon():
      global i
      while True:
        i += 1
        print(i)
        sleep(3)
    
    def application(environ, start_response):
      start_response('200 OK', [('Content-Type','text/html')])
      return [str(i).encode()]
    

    Or use:

    [uwsgi]
    master = false