Get live data continuosly from API using Flask

I am in the process of building my first basic website using Flask in Python. I have successfully created a website where I retrieve values from an API (specifically, ThingSpeak), and I display them in a table. My current challenge is figuring out if it's possible to obtain live data from the API without having to rerun my code.

As it stands, each time I run the code, I fetch the values from the API. However, I am looking for a way to continuously receive updated values without the need to manually rerun the code. Is there a method or approach to achieve this real-time data update within my Flask application?

Solution

The basic idea is that you add a Javascript function (marked 1 in the code) into your HTML file that sets an interval timer (2 in the code) for, say, every 3 seconds. When that interval passes, a function (3) is called. In that function, you make an AJAX request (4 in the code) to your flask (5 in the code) app. Your flask app gathers the latest data and sends it back to the Javascript function (6 in the code) which can then append a new line to the webpage with the latest data (7 in the code).

I have put all the Javascript, HTML and Python in a single file and avoided all CSS, Jinja templating and error-handling so you can see how it actually works. You can add fancy styling yourself.

#!/usr/bin/env python3

from flask import Flask, jsonify, request, url_for, redirect, session
app = Flask(__name__)

# This is the HTML, normally you would abstract it out into a template file and have Jinja render it
HTML = """
<!DOCTYPE html>
<html>
<head>
 <title>Continuous Update Demo</title>
 <!-- Pull in jQuery from Google-->
 <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
</head>

<body>
 <div>
   <h2>A continuously updated list:</h2>
   <ul id="culist">
   </ul>
 </div>

<script type="text/javascript">

function requestUpdate() {                         // 3

   console.log("DEBUG: requesting update");

   $.getJSON('/_refresh', function(result) {       // 4
      console.log("DEBUG: Received " + result.data);
      var HTML = "<li>" + result.data + "</li>";
      $("#culist").append(HTML);
   });

}

// This fires just once when the site initially loads
$(document).ready(function() {                      // 1
    console.log("DEBUG: document ready");
    // Get first element for list
    requestUpdate();
    // Ensure list is updated every 3s
    setInterval(requestUpdate, 3000);               // 2
});

</script>

</body>
</html>
"""

@app.route('/_refresh')                                # 5
def refresh():
    # print(request.args) to see what the Javascript sent to us
    print('DEBUG: Refresh requested');


    # Query your Thingspeak here and then pass the new data back to Javascript below


    # Build our response (a Python "dict") to send back to Javascript
    response =  { 
        'status' : 'Success', 
        'data': "A new line derived from Thingspeak",
    }
    print(f'DEBUG: Sending response {response}')
    return jsonify(response)                          # 6 - send the result to client


@app.route('/')
def main():
    print(f'DEBUG: Sending main page')
    return HTML

if __name__ == '__main__':
    app.run(debug=True)

Run this code with your web-browser's console turned on - it will be in your browser's "Developer Tools" or "Debug Tools" settings, then you will see all the messages.

You can obviously play around and tailor it to your own needs:

change the update rate,
CSS-style it better,
add newest lines at top instead of bottom,
cap the length of the displayed list and remove older elements as new ones arrive,
handle the situation when there is no new data available from Thingspeak - maybe by changing the status: "Success" in the returned JSON object to status: "Unchanged"
and so on...

Another, altogether different approach might be to open a web socket between flask and the Javascript. All the updates could then be initiated from the flask end whenever Thingspeak has an update, instead of the Javascript side checking every N seconds. That approach is described in my other answer.

Here is a little video of it running. You can see the (very un-styled) webpage running at the top, the Javascript console below that and the output from flask below that in the black Terminal.