Search code examples
pythonasynchronousplonezopepypi

Asynchronous tasks in Plone to query Python Package Index

I want to periodically (every hour?) query the Python Package Index API from Plone. Something equivalent to:

$ for i in `yolk -L 24 | awk '{print $1}'` # get releases made in last 24 hours
do
  # search for plone classifier
  results=`yolk -M $i -f classifiers  | grep -i plone`
  if [ $results ]; then
    echo $i
  fi
done

Results:

collective.sendaspdf
gocept.selenium
Products.EnhancedNewsItemImage
adi.workingcopyflag
Products.SimpleCalendarPortlet
Products.SimpleCalendar

Then I want to display this information in a template. I would love to, at least initially, avoid having to persist the results.

How do I display the results in a template without having to wait for the query to finish? I know there are some async packages available e.g.:

But I'm not sure what the general approach should be (assuming I can schedule an async task, I may need to store the results somewhere. If I have to store the results, I'd prefer to do it "lightweight" e.g. annotations)

Solution

  • How about the low, low tech version?

    Use a cron-job to run the query, put this in a temp file, then move the file into a known location, with a timestamp in the filename.

    Then, when someone requests the page in question (showing new packages), simply read the newest file in that location:

    filename = sorted(os.listdir(location))[-1]
data = open(os.path.join(location, filename)).read()

    By using a move, you guarantee that the newest file in the designated location is always a complete file, avoiding a partial result being read.