python irc bot trying to consume two different messaging system

Question

I'm using the python irc module [1] and the pika module in python to create an irc bot that listens to both channel messages and to a rabbitmq queue.

I took the source code from [2] and added the pika elements to it:

#! /usr/bin/env python
#
# Example program using irc.client.
#
# This program is free without restrictions; do anything you like with
# it.
#
# Joel Rosdahl <joel@rosdahl.net>

import sys
import argparse
import itertools

import irc.client
import pika


target = "#test"
"The nick or channel to which to send messages"


def on_connect(connection, event):
  if irc.client.is_channel(target):
      connection.join(target)
      return
  main_loop(connection)


def on_join(connection, event):
  main_loop(connection)


def get_lines():
  while True:
      yield sys.stdin.readline().strip()


def main_loop(connection):
  for line in itertools.takewhile(bool, get_lines()):
      print(line)
      connection.privmsg(target, line)
  connection.quit("Using irc.client.py")


def on_disconnect(connection, event):
  raise SystemExit()


def get_args():
  parser = argparse.ArgumentParser()
  parser.add_argument('server')
  parser.add_argument('nickname')
  parser.add_argument('target', help="a nickname or channel")
  parser.add_argument('-p', '--port', default=6667, type=int)
  jaraco.logging.add_arguments(parser)
  return parser.parse_args()


def callback(ch, method, properties, body):
  print(" [x] Received %r" % body)


def get_channel():
  creds = pika.PlainCredentials('testuser', 'testing')

  params = pika.ConnectionParameters(
      host="localhost",
      virtual_host="/test",
      credentials=creds)

  connection = pika.BlockingConnection(params)
  channel = connection.channel()

  channel.queue_declare(queue='test')

  channel.basic_consume(
      queue='test', on_message_callback=callback, auto_ack=True)

  return channel


def main():
  chan = get_channel()

  reactor = irc.client.Reactor()
  try:
      c = reactor.server().connect("irc.local", 6667, "testuser")
  except irc.client.ServerConnectionError:
      print(sys.exc_info()[1])
      raise SystemExit(1)

  c.add_global_handler("welcome", on_connect)
  c.add_global_handler("join", on_join)
  c.add_global_handler("disconnect", on_disconnect)

  print("Processing reactor")
  reactor.process_forever()
  print("Channel : start consuming")
  channel.start_consuming()


if __name__ == '__main__':
  main()

The problem with the above code is that I haven't modified the get_lines() code to actually get from the messaging queue as I got stuck with what to change it for.

Also, the 'reactor.process_forever()' line blocks the 'channel.start_consuming()' line and obviously, if I move the channel.start_consuming() above the reactor.process_forever(), the reactor.process_forever() doesn't run.

At this point, I'm stumped. I thought about using multiprocessing threads; but my experience with threads is nil and even after reading [3], I'm not entirely sure that's going to help. To be honest, it confused me a bit more.

I thought of adding an on_* callback handler but since those events are all irc based, the handler wouldn't be listening to the rabbitmq queue.

Might anyone have a suggestion in how to both run the process_forever() loop and the start_consuming() loop; that is, to get the bot to listen to the irc channel and the messaging queue?

Thanks!

:ed

[1] - https://github.com/jaraco/irc

[2] - https://github.com/jaraco/irc/blob/master/scripts/irccat.py

[3] - https://realpython.com/intro-to-python-threading/

Answer 1

Thanks to @fura (so kudos!) for the help in clarifying what I can do. The final working resulting code is as follows:

#! /usr/bin/env python
#
# Example program using irc.client.
#
# This program is free without restrictions; do anything you like with
# it.
#
# Joel Rosdahl <joel@rosdahl.net>

import sys
import argparse
import itertools

import irc.client
import pika


target = "#test"
"The nick or channel to which to send messages"


def on_connect(connection, event):
    if irc.client.is_channel(target):
        connection.join(target)
        return


def on_disconnect(connection, event):
    raise SystemExit()


def get_channel():
    creds = pika.PlainCredentials('testuser', 'testing')

    params = pika.ConnectionParameters(
        host="msg.local",
        virtual_host="/test",
        credentials=creds)

    connection = pika.BlockingConnection(params)
    channel = connection.channel()

    channel.queue_declare(queue='test')

    return channel


def main():
    chan = get_channel()

    reactor = irc.client.Reactor()
    try:
        print("connect to server")
        c = reactor.server().connect("irc.local", 6667, "testUser")
    except irc.client.ServerConnectionError:
        print(sys.exc_info()[1])
        raise SystemExit(1)

    c.add_global_handler("welcome", on_connect)
    c.add_global_handler("disconnect", on_disconnect)

    print("Processing reactor")
    while True:
        reactor.process_once()
        mf, hdr, bd = chan.basic_get("test")
        if mf:
            chan.basic_ack(mf.delivery_tag)
            bdd = bd.decode('utf-8')
            if "cmd:" in bdd:
                p = bdd.replace("cmd:", "").strip()
                if p.lower() == "quit":
                    c.quit("Buckeroo Banzai!")
                else:
                    c.privmsg("#test", bdd)
            else:
                c.privmsg("#test", bdd)


if __name__ == '__main__':
    main()

Of course, I tested this with short messages.. haven't sent a huge text file through that, so not sure if it is efficient or wastes resources. Will need to put it in practice to see if it chokes.

Again, @fura, appreciate the help and advice!

:Ed