Synchronisation of data from different sensor

Question

So we have a system which detects the activity of a human, he have multiple sensor on him (accelerometer,etc...) and each sensor learn individually with machine learning algorithm. So a sensor can send : "I think at 80% the man is standing" for example. For now we have made this with dataset synchronised but in real life the sensors are asynchronous and send the data at different time (the clocks are different on each sensor). So the question is how we can make a dataset "synchronised" from a dataset with empty data.

Exemple (each 50ms a row):

So how can I fill the empty cells by a value to have response from all sensors because we have a fusion function which says when we have a row of data which activity the man is doing.

I hope you understand what I am searching. I think we need some mathematical function to fill the empty cells like a linear function or other but I need to be sure and I don't find a research report on the web which confirm that or tell me what function to use.

Answer 1

As I see it, it depends on what frequency you want the output at. From your post it seems you seem to want to be running at the fastest sensor clock.

The point of having multiple sensors is to increase the amount of information available. By filling out the "missing values" from information that you already have, you will most likely end up with more computing for a result that has no or close to no added value.

Here are two solutions I can think of, one simple and the other one a bit more complex depending on your level, what platform you use and the libraries available on it.

1. Waiting for all sensors

A simple solution would be to just wait for all (or part of the) sensors to have sent a value, before launching processing for fusion. This provides unambiguous results, but "low" framerate

2. Kalman filter

If you are willing to go into more depth to get these values, get a faster refresh rate, and don't mind spending some processing power, you can use some prediction filter, like a Kalman filter, on each sensor to estimate its value based on its previous evolution at a time dt where you have not yet received the value from the real sensor. This works only if the duration over which you estimate is very short relative to the variation of their values. This also allows you to go beyond the speed of the fastest sensor. Here is a beginner friendly explanation of the Kalman filter.