Search code examples
pythonjsontxt

How to save multiple file paths in a txt file for json.load() to read?


I have two json files that I'd like to open to read from and instead of calling the open() function twice to open the json files individually, I tried this:


with open('./file_paths.txt') as file_list:
    for filename in file_list:
        json_data = json.load(file_list)

where in the './file_paths.txt' file I've got two paths listed like this:

./dataset_1.json,
./dataset_2.json

I have a comma separating them. I tried enclosing them in quotation marks but I'm still getting the same error:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

There is a problem with this line:

json_data = json.load(file_list)

in that I think the problem is how I've listed the filepaths in the .txt file.


Solution

  • The documentation for the json.load function (https://docs.python.org/3/library/json.html) refers to a single file or file-like object:

    json.load(fp, *, cls=None, object_hook=None, parse_float=None,parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

    Deserialize fp (a .read()-supporting text file or binary file containing a JSON document) to a Python object using this conversion table.

    As you seem to want the loaded data from two files combined into one object, the answer here (https://stackoverflow.com/a/66059974/4438898) looks useful. It creates a 'merge_dict' function that combines two json-derived dictionaries into one, merging the entries where the keys are identical.

    Reading a list of files and loading the json as a list of dicts

    (Added in response to your replies)

    This reads lines from mylistfile.txt line by line, strips the trailing '\n' from each line, then loads the json data from the resulting filename. The json data is added as a dictionary to the results list, and the results are printed:

    Adding data from dataset_1.json
    Adding data from dataset_2.json
    Results: [{'hello': 1, 'there': 2}, {'key1': 1, 'key2': 2}]
    

    mylistfile.txt was a 2 line file:

    dataset_1.json
    dataset_2.json