Threads in twisted... how to use them properly?

Question

I need to write a simple app that runs two threads: - thread 1: runs at timed periods, let's say every 1 minute - thread 2: just a 'normal' while True loop that does 'stuff'

if not the requirement to run at timed interval I would have not looked at twisted at all, but simple sleep(60) is not good enough and construction like:

l = task.LoopingCall(timed_thread)
l.start(60.0)
reactor.run()

Looked really simple to achieve what I wanted there.

Now, how do I 'properly' add another thread?

I see two options here:

• Use threading library and run two 'python threads' one executing my while loop, and another running reactor.run(). But Google seems to object this approach and suggests using twisted threading
• Use twisted threading. That's what I've tried, but somehow this looks bit clumsy to me.

Here's what I came up with:

def timed_thread():
    print 'i will be called every 1 minute'
    return

def normal_thread():
    print 'this is a normal thread'
    time.sleep(30)
    return

l = task.LoopingCall(timed_thread)
l.start(60.0)
reactor.callInThread(normal_thread)
reactor.run()

That seems to work, but! I can't stop the app. If I press ^C it wouldn't do anything (without 'callInThread' it just stops as you'd expect it to). ^Z bombs out to shell, and if I then do 'kill %1' it seems to kill the process (shell reports that), but the 'normal' thread keeps on running. kill PID wouldn't get rid of it, and the only cure is kill -9. Really strange.

So. What am I doing wrong? Is it a correct approach to implement two threads in twisted? Should I not bother with twisted? What other 'standard' alternatives are to implement timed calls? ('Standard' I mean I can easy_install or yum install them, I don't want to start downloading and using some random scripts from random web pages).

Answer 1

Assuming that your main is relatively non-blocking:

import random
from twisted.internet import task

class MyProcess:
  def __init__(self):
    self.stats = []
    self.lp = None
  def myloopingCall(self):
    print "I have %s stats" % len(self.stats)
  def myMainFunction(self,reactor):
    self.stats.append(random.random())
    reactor.callLater(0,self.myMainFunction,reactor)
  def start(self,reactor):
    self.lp = task.LoopingCall(self.myloopingCall)
    self.lp.start(2)
    reactor.callLater(0,self.myMainFunction,reactor)
  def stop(self):
    if self.lp is not None:
      self.lp.stop()
    print "I'm done"

if __name__ == '__main__':
  myproc = MyProcess()
  from twisted.internet import reactor
  reactor.callWhenRunning(myproc.start,reactor)
  reactor.addSystemEventTrigger('during','shutdown',myproc.stop)
  reactor.callLater(10,reactor.stop)
  reactor.run()

$ python bleh.py
I have 0 stats
I have 33375 stats
I have 66786 stats
I have 100254 stats
I have 133625 stats
I'm done