I am new to InfluxDB and having doubts about how to store my data. We have an output from our optical sensors that are thousands values per timestamp. So far I've seen examples only for few values per timestamp. My data would be something like this:
timestamp, tags{}, fields{value1, value2... value10000}
Is there a best practice to store data like this in InfluxDB? Also, is there a limit for how many fields one data point can have?
Without knowing the exact relationship between the sensors and their data, here are 3 potential options.
Case 1: Time Sequential Data
It's worth noting that InfluxDB is able to use timestamps down to nanosecond precision. If your thousands of values are sequential and can be individually timestamped, consider writing/storing them in individual timestamps
INSERT measurement sensor=[value1] 2016-11-26T00:00:00.00000000Z
INSERT measurement sensor=[value2] 2016-11-26T00:00:00.00010000Z
...
INSERT measurement sensor=[value10000] 2016-11-26T00:00:01.00000000Z
This has the advantage of allowing all your measurements to be seen in one time series.
Case 2: One-to-one mapping sensor to value
If each value comes from a unique sensor such that you have thousands of points and thousands of sensors, consider making a tag or field for each sensor.
A: Tagging values by sensor
INSERT measurement,sensor=sensor1 field=[value1] 2016-11-26T00:00:00Z
INSERT measurement,sensor=sensor2 field=[value2] 2016-11-26T00:00:00Z
...
INSERT measurement,sensor=sensor10000 field=[value10000] 2016-11-26T00:00:00Z
B: Each sensor is a field
INSERT measurement sensor1=[value1] sensor2=[value2] ... sensor10000=[value10000]
If there are multiple metrics coming from each sensor I would consider option A. If each sensor only produces 1 metric, I would choose option B.
Case 3: Variable number of values
If you have a variable number of values per timestamp, for example, in machine vision where you've thresholded and extracted an object from an image, you are hitting the limits of a TSDB. Spatial data like this does not map well into a metric oriented based time series database like InfluxDB.