Search code examples
mongodbtime-seriessensorsiotbigdata

Timestamp as Key in mongodb as timeseries database

I want to use mongodb as timeseries database and query via timestamp + id.

Mongodb as shown a way to store data here.

{
  timestamp_hour: ISODate("2013-10-10T23:00:00.000Z"),
  ID: “System1”,
  values: {
    0: { 0: 999999, 1: 999999, …, 59: 1000000 },
    1: { 0: 2000000, 1: 2000000, …, 59: 1000000 },
    …,
    58: { 0: 1600000, 1: 1200000, …, 59: 1100000 },
    59: { 0: 1300000, 1: 1400000, …, 59: 1500000 }
  }
}

but i have multiple values and for every value a timestamp, its not periodic. Data has some time a delay or is not coming for days. So i dont want to use 0-24 for my hours and 0-59 for my minutes. Can i use instead my measured timestamp? every value in my document has the same timestamp, so if value1 has 50 entries , value2 has also 50 entries and equal timestamp. 

     {
          timestamp_hour: ISODate("2013-10-10T23:00:00.000Z"),
          ID: “System1”,
          values1: {
            "2013-10-10T22:00: {
         "2013-10-10T22:01:00.000Z": 999999, 
         "2013-10-10T22:02:00.000Z": 999999,
         "2013-10-10T22:03:00.000Z": 1000000 
                                },
            "2013-10-10T23:00:": { 
         "2013-10-10T23:01:00.000Z": 2000000,
         "2013-10-10T23:02:00.000Z": 2000000, 
                                 },
          }
         values2: {
            "2013-10-10T22:00: {
         "2013-10-10T22:01:00.000Z": 999999, 
         "2013-10-10T22:02:00.000Z": 999999,
         "2013-10-10T22:03:00.000Z": 1000000 
                                },
            "2013-10-10T23:00:": { 
         "2013-10-10T23:01:00.000Z": 2000000,
         "2013-10-10T23:02:00.000Z": 2000000, 
                                 },
         }
    }
Solution

  • You could store your values with the incoming timestamp, and use a separate mongodb map-reduce process to transform your values and store them in different collections with the granularity you need (ex. hourly, daily, monthly). See also incremental map-reduce.

    You could also find these posts inspiring: