How to insert historical data into fiware (with correct dates)?

Question

I have a bunch of historical data (csv) which I want to make accessible through sth-comet. The data is the history of water levels from multiple revers. The data is not provided live, but more or less on a daily basis and contains all the historic records for multiple days.

What I did so far was:

1. Convert the data into NGSIv2 format data model with dateObserved: DateTime and waterlevel : number field
2. Update/append the data into Fiware orion
3. Create a subscription for sth-comet for the entity type
4. Access the historical data in sth-comet (wrong time)

With this I now have the problem that the "rcvTime" is of course the time when sth-cometreceived the data. Is there a way that I can "overwrite" that attribute or is there a better solution? I also looked at cygnus on inserting data but I think the underlying problem is the same. I could not find any hint in the avaiable documentation.

Answer 1

In the case of using Cygnus NGSIMongoSink and NGSISthSink you can use TimeInstant metadata in attributes to override received time with the time given in the metadata value.

Have a look to NGSIMongoSink documentation

By default, NGSIMongoSink stores the notification reception timestamp. Nevertheless, if (and only if) working in row mode and a metadata named TimeInstant is notified, then such metadata value is used instead of the reception timestamp. This is useful when wanting to persist a measure generation time (which is thus notified as a TimeInstant metadata) instead of the reception time.

or this similar fragment in NGSISTHSink documentation:

By default, NGSISTHSink stores the notification reception timestamp. Nevertheless, if a metadata named TimeInstant is notified, then such metadata value is used instead of the reception timestamp. This is useful when wanting to persist a measure generation time (which is thus notified as a TimeInstant metadata) instead of the reception time.