Search code examples
pythonmultithreadingipcprocessinteraction

How to make two python programs interact?

I have a HTTP sever in one program and my basic application in another one. Both of them are loops, so I have no idea how to:

  1. Write a script that would start the app and then the HTTP server;
  2. Make these programs exchange data in operation.

How are these things usually done? I would really appriciate Python solutions because my scripts are written in Python.

  1. Does a user make an http request which queries the app for some data and return a result? Yes

  2. Does the app collect data and store it somewhere? The app and the HTTP Server both use SQLite database. However the DBs may be different.

Solution

  • Before answering, I think we need some more information:

    1. Is there a definable pipeline of information here?
      1. Does a user make an http request which queries the app for some data and return a result?
      2. Does the app collect data and store it somewhere?

    There are a few options depending on how you're actually using them. Sockets is an option or passing information via a file or a database.

    [Edit] Based on your reply I think there's a few ways you can do it:

    1. If you can access the app's database from the web server you could easily pull the information you're after from there. Again it depends what information it is that you want to exchange.
    2. If your app just needs to give the http server some results, you could write them into a results table in the http server's db.
    3. Use pipe's or sub processes as other people have suggested to exchange data with the background app directly.
    4. Use a log file which your app can write to and your http server read from.

    Some more questions:

    1. Do you need two-way communication here or is the http server just displaying results?
    2. What webserver are you using?
    3. What processing languages do you have available on it?

    Depending on how reliant the two parts can be, it might be best to write a new app to check the database of your app for changes (using hooks or polling or whatever) and post relevent information into the http server's own database. This has the advantage of leaving the two parts less closely coupled which is often a good thing.

    I've got a webserver (Apache 2) which talks to a Django app using the fastcgi module. Have a look at the section in djangobook on fastcgi. Apache uses sockets (or regular tcp) to talk to the background app (Django).

    [Edit 2] Oops - just spotted that your webserver is a python process itself. If it's all python then you could launch each in it's own thread and pass them both Queue objects which allow the two processes to send each other information in either a blocking or non-blocking manner.