Search code examples
pythonrequestremote-serverjob-scheduling

How to get status of a job running on another machine?


I have 2 machines A and B and A can send restful request to B as follows:

curl -XPOST -H "Content-type: application/json" -d '{"data":"python /tmp/demo.py","action":"demo"}' 'http://192.168.95.8:51888/api/host'

I have deployed an api service on B and when such request is received, B will execute the python script /tmp/demo.py and the execution may last 0.5-3 hours.

My question is:

1) How to write a job on A that keeps tracking the status of the task running on B and end it self when the task finishes successfully or failed?

2) In the tracking job, how to add a module that can kill itself after exceeding a pre-set time threshold?


Solution

  • Treat the job as an HTTP resource. When you do POST /api/host, that request creates a new id for that job and returns it. For good use of HTTP, the response would contain a Location header with the URL of the resource where the job's status can be checked, e.g.:

    POST /api/hosts
    Content-type: application/json
    
    {"data":"python /tmp/demo.py","action":"demo"}
    
    HTTP/1.1 201 Created
    Location: /api/host/jobs/c2de232b-f63e-4178-a053-d3f3459ab538
    

    You can now GET /api/host/jobs/c2de232b-f63e-4178-a053-d3f3459ab538 at any time and see what status the job has, e.g.:

    {"status": "pending"}
    

    You may POST commands to that resource, e.g. for cancelling it.

    How exactly your HTTP API would get the status of that Python script is obviously up to you. Perhaps it can communicate with it over a socket, or the job itself will periodically write its status to some database or file.