Search code examples
javascriptjsongoogle-chrome-extensionfirefox-addon

What's the most efficient way to get a key from a huge json?


I'm making a dictionary browser extension that for various reasons can't use an online API. Instead, I have two huge .json files, one for English to the target language and the other for vice versa, the larger of which is about 26MB. I'm aware that I can use fetch() to import a json file as an object, but I'm afraid that something that large will heavily impact performance.

Is there a way to retreive just one key from a huge .json file in js? Would it be best if I hosted the .json online somehow? Or is 26MB actually not such a big deal, and I should just import the whole thing as an object?


Solution

  • In short: You probably need to load the whole JSON.

    JSON

    JSON is primarily intended to be used as a transfer format, i.e. the result of one fetch request would be one JSON file.

    JSON is just a text file, so there is no way to "just get one key". There are no "keys" yet.

    keys and values

    You will have "keys" (and values) as soon as the JSON is converted into data that can be used by some code. This can be done in many various ways, e.g.:

    • Convert the JSON into a javascript object, and find the keys with javascript
    • Put the data into a database, and find the keys with database functions

    Storing data

    Usually you should store large data in a database, like IndexedDB in the browser, or a Mongo-Db, MySql-Db, ... on the server, and query the wanted data with the functions provided by the DB.

    If you can't use a database, you still need a data representation (instead of a JSON string), e.g. load the whole data into the memory, e.g. a javascript object on the client, or e.g. a PHP array on the server.

    Conclusion

    If the server can't provide selected parts of the data, and you can't modify the server, then the only thing that you can do is load the whole JSON string, convert it to javascript, and find the keys on the client.

    (You could split the JSON data into separate files, as others suggested. But going that way, together with necessary additional code, you would in essence start to build an own, simple, very limited "database".)