Is there any way to make my Mcp3008 sampling uniform?

Question

I'm doing a job to use ADC mcp3008 sample sensor data and collect them until 10k.Then send it to influxdb.

All these work should be done in 1 second. That's the point.

Now the problem is, the timestamp of each data are very uneven.

As you can see:

I want the timestamp be uniform at 0.1ms. I mean the time precision in influsdb should be 0.1ms.

But unfortunately, the write parameter time_precision only have 's', 'ms', 'u' or 'n'.

So all I can do is make the sampling process more uniform, right?

I use multiprocessing module to do this work. And here is my original Code:

import Adafruit_GPIO.SPI as SPI # Import Adafruit GPIO_SPI Module
import Adafruit_MCP3008         # Import Adafruit_MCP3008
import serial
import time
import datetime
from influxdb import InfluxDBClient
from multiprocessing import Process, Queue
def producer(name):
    i=0
    while True:
        begin=time.time()
        body = []
        while i<10000:
            val = round(mcp.read_adc(0),4) #here read the data from SPI port
            current_time = datetime.datetime.utcnow()
            js = {
                "measurement": "Double",
                "time": current_time,
                "tags": {
                },
                "fields": {
                    "sensor2": val
                }
            }
            body.append(js)
            i+=1
        i=0
        res = client.write_points(body) #Send influxdb 10k data at once
        body.clear()
        end=time.time()-begin
        print(end,name)
           
if __name__ == "__main__":
    HW_SPI_PORT = 0 # Set the SPI Port. Raspi has two.
    HW_SPI_DEV  = 0 # Set the SPI Device
    mcp = Adafruit_MCP3008.MCP3008(spi=SPI.SpiDev(HW_SPI_PORT, HW_SPI_DEV))
    client = InfluxDBClient(host='XXXXX', port=8086, username='admin', password='admin', database= 'db',ssl=False, verify_ssl=False)
    p1 = Process(target=producer,args=(0,))
    p2 = Process(target=producer,args=(1,))
    p3 = Process(target=producer,args=(2,))
    p4 = Process(target=producer,args=(3,))
    p5 = Process(target=producer,args=(4,))
    p6 = Process(target=producer,args=(5,))
    p1.start()
    p2.start()
    p3.start()
    p4.start()
    p5.start()
    p6.start()

Yeah...I had to go through six processes to finish in an average of one second..

So is there any way to make the sampling uniform? To make the timestamp like this:

1603469938916'5'26000   -0.175
1603469938916'6'26000   -0.172
1603469938916'7'26000   -0.178
1603469938916'8'26000   -0.175
1603469938916'9'26000   -0.182

I mean with the time precision 0.1ms.

Thanks! This must be a strange problem.

PS: I got an idea, is there any way to make my timestamp’s precision be 0.1ms? Something like :

timestamp=datetime.datetime.utcnow()
...Some operation...
print(timestamp)

Then get: 1603469938916900000

It may work.

Yeah, I find the solution:

from datetime import datetime
import math
def format():
    dt = datetime.utcnow()
    dt_round_microsec = math.floor(dt.microsecond/100)*100 
    dt = dt.replace(microsecond=dt_round_microsec)
    return dt

Better proposals are welcome

Answer 1

A couple suggestions.

1. If you need to sample the ADC uniformly, this is a harder problem.
2. If you "just need" uniform timestamp spacing, you can sample however you like and then set the timestamps to begin + iteration * 0.1ms. This option is not great for any sort of scientific data collection. 10k samples/sec sounds like you need evenly spaced sampling (e.g. for audio). So I will ignore option 2.

For option 1, you need your loop to have a consistent iteration time. This is very hard to guarantee with python on a RPi (I assume you're using a Pi). The RPi OS is not real time so your loop may be delayed randomly. The only real option here is to use an external microprocessor to trigger the ADC with guaranteed timing.

We can try to make your sampling loop as written somewhat better though. datetime.datetime.utcnow() may be a slow-ish system call. You may get better results with time.perf_counter() instead. You may also want to store just val and current_time in lists within the loop and assemble the full json body outside the loop in a different process since it looks like you want 10ksps forever (and send to influxdb in a separate process too).

In general, the raspberry pi hardware is not setup to do continuous real-time sampling of an ADC at these frequencies.