At a high level, what I need to do is have a python script that does a few things based on the commands it receives from various applications. At this stage, it's not clear what the application may be. It could be another python program, a MATLAB application, or a LAMP configuration. The commands will be sent rarely, something like a few times every hour.
The problem is - What is the best way for my python script to receive these commands, and indicate to these applications that it has received them?
Right now, what I'm trying to do is have a simple .txt file. The application(s) will write commands to the file. The python script will read it, do its thing, and remove the command from the file.
I didn't like this approach for 2 reasons- 1) What happens if the file is being written/read by python and a new command is sent by an application? 2) This is a complicated approach which does not lead to anything robust and significant.
There are few integration patterns usable for this purpose.
Messaging allows decoupling by communicating contracted messages, all the rest is left as platform specific detail, which does not affect other parties.
Other method could be e.g. shared data store - one side would store requests for processing there, another would fetch them and process.
ZeroMQ is a library, allowing very lightway messaging without requiring to run some big messaging application.
What you describe could be resolved this way:
With clients being in multiple languages you shall use some cross-platform serialization, result buffers or even better JSON strings are good options.
ZeroMQ has libraries for many languages, so you shall be able to communicate.
There is great ZeroMQ Guide providing clear explanation of concepts and showing implementation in many languages.
For planning the communication infrastructure, I would start with your Python script being bound to some TCP port (being "fixed" part) and your clients connecting to it.
Python script would use REP socket (for "reply"), your clients in other languages REQ sockets.
Following samples come from huge set of examples in zguide written for many languages
hwserver.py):import zmq
import time
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")
while True:
# Wait for next request from client
message = socket.recv()
print("Received request: %s" % message)
# Do some 'work'
time.sleep(1)
# Send reply back to client
socket.send(b"World")
hwclient.php):<?php
/*
* Hello World client
* Connects REQ socket to tcp://localhost:5555
* Sends "Hello" to server, expects "World" back
* @author Ian Barber <ian(dot)barber(at)gmail(dot)com>
*/
$context = new ZMQContext();
// Socket to talk to server
echo "Connecting to hello world server...\n";
$requester = new ZMQSocket($context, ZMQ::SOCKET_REQ);
$requester->connect("tcp://localhost:5555");
for ($request_nbr = 0; $request_nbr != 10; $request_nbr++) {
printf ("Sending request %d...\n", $request_nbr);
$requester->send("Hello");
$reply = $requester->recv();
printf ("Received reply %d: [%s]\n", $request_nbr, $reply);
}
Shared file or database also allows communication between processes. The problem becomes complex with locking resources, taking care, that the same task is not processed by multiple workers etc.
There are multiple patterns and multiple platforms to implement.