Search code examples
node.jsmongodbmongoosetime-series

Can't seem to create a time series collection on insert with mongoose,nodejs


I am using mongoose to create a schema. I want to insert data into a time series collection. I want to insert and create the collection at the same time. So I created my mongoose schema which looks like this:

const mongoose = require('mongoose')
const Schema = mongoose.Schema;

const pmSchema = new Schema({
    aqius: Number,
    conc: Number
})
const MeasurementSchema = new Schema({
    ts: Date,
    pm1: Number,
    pr: Number,
    hm: Number,
    tp: Number,
    pm25: pmSchema,
    pm10: pmSchema,
});

const DailySchema = new Schema({
    time: Date,
    daily: [{
        type: MeasurementSchema,
    }],
    name: String,
}, {
    timeseries: {
        timeField: 'time',
    }
});

module.exports = mongoose.model('DailyMeasurement', DailySchema, 'daily-collection-ts')

I then use this schema in a cron job which is just a separate js file which fetches the data and saves it in the mongoDB. The cron job looks like this:

const {MongoClient} = require("mongodb");
require('dotenv').config();
const DailyMeasurementModel = require('../app/Models/AQDataDaily');
const tokens = [
    process.env.aaaaa,
    process.env.ssssss,
    process.env.dddddd,
]
const urls = tokens.map(token => process.env.COMMON_DEVICE_URL + token)
const data_files = [
    '../test-data-json/asdasdasdasd.json',
    '../test-data-json/aaaa.json',
    '../test-data-json/sssss.json',
]
const getData = async () => {
    try {
        let data = []
        for (let i = 0; i < data_files.length; i++) {
            //FETCH DATA FROM JSON FILES FOR TESTING
            const response = require(data_files[i]);
            data.push(response)
            await saveData(data[i]);
          
        }
    } catch (error) {
        console.log('Error at getData: ', error);
    }
}

async function saveData(data) {
    const client = new MongoClient(process.env.MONGO_COMPASS_URI, {useUnifiedTopology: true});
    try {
        await client.connect();
        const db = client.db("iq-air-database");
        const coll = db.collection("daily-collection-ts");
        const timestamp = new Date().getTime();
        const newDailyMeasurement = new DailyMeasurementModel({
            time: timestamp,
            daily: data.historical.daily,
            name: data.name
        })

        try {
            // Save newDayMeasurement instance to database
            const result = await coll.insertOne(newDailyMeasurement)
            console.log('Document saved:', result);
        } catch (err) {
            console.error('Error saving document:', err);
        }
    } catch (error) {
        console.log('Error at the start of saveData: ', error);
    } finally {
        await client.close();
    }
}

getData()

The data is saved correctly in the mongoDB but the collection is a regular collection not a time series collection. I checked my mongo version it is high enough. I also used the mongoDB compass shell to create a time series collection manually. The expected result is that the data is saved in a time series collection instead of a regular collection like it is being saved right now.


Solution

  • The links provided by @user200492973 (How to create Time Series Collection with Mongoose) Did provide the solution. However it seems to me that sometimes a time series collection is created and in other times a regular collection is created.

    I can't figure out why.