How can I optimize data retrieval in my KTOR backend server when processing API requests?

Question

I am building a KTOR backend server which pull data from an API and stores it in MongoDB Atlas.Some data which is always same I have stored it in MongoDB.I need it frequently to process API data. I have two option? Either I call pull it from mongo at every API request, which will increase may connections to DB, Or what I have done store that data in JSON file in resourses folder of code, and pull required data from that file.

Is this second approach is better than first in terms of performance?

Note- that json document is array of 10000 objects, 1.5MB in size.

This is the code I am using to fetch data

fun getStationsFromJson(statCode:String): Station? {
    val jsonText = File("fillepath").readText()
   val stations =Json.decodeFromString<List<Station>>(jsonText)
    val foundStationByCode = stations.find { it.stationCode == statCode }

    return foundStationByCode
}

Is this approach good? Or are there anything else I can do?

Answer 1

You can try caching your JSON data in memory upon server startup, using a HashMap for efficient O(1) lookups by stationCode

sample

object StationCache {
    private val stationMap: Map<String, Station>

    init {
        val jsonText = File("filepath").readText()
        val stations = Json.decodeFromString<List<Station>>(jsonText)
        stationMap = stations.associateBy { it.stationCode }
    }

    fun getStationByCode(statCode: String): Station? {
        return stationMap[statCode]
    }
}

fun getStationsFromJson(statCode: String): Station? {
    return StationCache.getStationByCode(statCode)
}