Search code examples
pythonexport-to-csv

How to write to csv file every x minutes within while True loop


I have a loop that gets sensor information, makes the data available for a real-time dashboard, and also writes to a CSV file. I would like continue to make the real-time data available without any delay but only want to write to the CSV file every 5 minutes. Time.sleep() doesn't seem to work for this because it pauses everything. Any suggestions on how to approach this?

Here's the relevant loop:

while True:
        with open('enviroplus.csv', 'a', newline='') as csvfile:
            writer = csv.DictWriter(csvfile, delimiter=',', quotechar='|', 
quoting=csv.QUOTE_MINIMAL, fieldnames=['Date/Time','temperature','humidity','pressure','oxidising','reducing','nh3','lux','proximity','pm1','pm25',   'pm10'])                                                                                                                                                                            
            dt = datetime.now()
            if not file_exists:
                writer.writeheader()
                file_exists = True
            writer.writerow({'Date/Time': dt.strftime('%Y-%m-%d %H:%M:%S'), 
'temperature': ...})

    get_temperature(args.factor)
    get_pressure()
    get_humidity()
    get_light()
    if not args.enviro:
        get_gas()
        get_particulates()
    if DEBUG:
        logging.info('Sensor data: {}'.format(collect_all_data()))

Solution

  • Store the last updated time in a variable, then if five minutes have passed update it:

    last_updated = datetime.now() - timedelta(minutes=6)
    
    while True:
        if ((datetime.now() - last_updated).seconds / 60) >= 5:
            with open('enviroplus.csv', 'a', newline='') as csvfile:
                writer = csv.DictWriter(csvfile, delimiter=',', quotechar='|', 
    quoting=csv.QUOTE_MINIMAL, fieldnames=['Date/Time','temperature','humidity','pressure','oxidising','reducing','nh3','lux','proximity','pm1','pm25',   'pm10'])                                                                                                                                                                            
                dt = datetime.now()
                if not file_exists:
                    writer.writeheader()
                    file_exists = True
                writer.writerow({'Date/Time': dt.strftime('%Y-%m-%d %H:%M:%S'), 
    'temperature': ...})
            last_updated = dt
        get_temperature(args.factor)
        get_pressure()
        get_humidity()
        get_light()
        if not args.enviro:
            get_gas()
            get_particulates()
        if DEBUG:
            logging.info('Sensor data: {}'.format(collect_all_data()))
    

    Make sure you also import timedelta from datetime. Note that last_updated is initially set to six minutes ago to make sure that the file updates on the first iteration of the loop.