How do I create a file that doesn't contain information older than 5 minutes?

Question

I'm working on Twitter Streaming api to get live tweets data.

I can print that data to console. But what I want is to save the data to a file and that data shouldn't be older than 5 minutes.

How do I continuously roll the file that holds the data from last 5 minutes as we can do for log files.

At the same time the file should be accessible for reading.

Is there any way to do that in Python?

I haven't come across such thing where we can mention such duration for which the file can hold specific data.

Answer 1

A search for "python logrotate" finds this: https://docs.python.org/3.7/library/logging.handlers.html#logging.handlers.TimedRotatingFileHandler. It is probably worth giving a try.

If you'd rather code something yourself, managing the tweets in memory (with a collections.deque for example so it's easy to pop old ones and append new ones), then flush this to file once in a while..., (or use sockets, pickle, simply the variable name to pass this data to your analysis function, as already mentioned in other answers).