Search code examples
pythonjsonpython-3.xdictionarybinaryfiles

ValueError: malformed node or string with ast.literal_eval()

I'm simply trying to parse a binary file into Python. Here is the code for my program:

try:
    with open('saves/temp/data.bin', 'r') as file:
        confirmed_data = json.load(file)
except:
    pass

confirmed_mode = confirmed_data[0]
confirmed_location = confirmed_data[1]
confirmed_units_data = confirmed_data[2]
confirmed_units = ast.literal_eval(confirmed_units_data)
confirmed_unit_quantities_data = confirmed_data[3]
confirmed_unit_quantities = ast.literal_eval(confirmed_unit_quantities_data)

I tried the following as data.bin and it worked perfectly:

["PLANETARY", "Naboo", "{'Optimal': [], 'Default': ['B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids']}", "{'B1 Battle Droids': 100}"]

However, when I tried the data below it didn't work and I have no idea why since it is basically identical to the data that I used above, which worked successfully:

["PLANETARY", "Kamino", {"Optimal": ["Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids"], "Default": ["B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids"]}, {"Aqua Droids": 60, "B1 Battle Droids": 40}]

I've looked at similar SO questions and found nothing to fix my problem. I've checked if data.bin has anything that is not a Python literal (i.e. Decimal datatype or variable name) but found nothing. I'm not exactly sure why the first works and the second doesn't. I'm using Python 3.6.

Solution

  • As mentioned by Daniel Pryden in the comments, the data is different. Some parts of the first bin are quoted as strings, whereas the second bin is standard JSON.

    I first figured out a way to classify the two different files. I found that the fully JSON one was the default generated manually, and that the partially string one was generated from a config file.

    Therefore, I added a 'config' flag to the config file,

    ["PLANETARY", "Naboo", "{'Optimal': [], 'Default': ['B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids', 'B1 Battle Droids']}", "{'B1 Battle Droids': 100}", "config"]

    And a 'manual' flag to the default file,

    ["PLANETARY", "Kamino", {"Optimal": ["Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids", "Aqua Droids"], "Default": ["B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids", "B1 Battle Droids"]}, {"Aqua Droids": 60, "B1 Battle Droids": 40}, "manual"]

    I then used the following code to identify the config file one using the flag, and perform ast.literal_eval on the parts of the file that are strings. If the manual file is used, this is skipped and the standard JSON is used as-is.

    import ast

try:
    with open('saves/temp/data.bin', 'r') as file:
        confirmed_data = json.load(file)
except:
    pass

confirmed_mode = confirmed_data[0]
confirmed_location = confirmed_data[1]

if confirmed_data[4] == 'config':
    confirmed_units_data = confirmed_data[2]
    confirmed_units = ast.literal_eval(confirmed_units_data)
    confirmed_unit_quantities_data = confirmed_data[3]
    confirmed_unit_quantities = ast.literal_eval(confirmed_unit_quantities_data)
else:
    confirmed_units = confirmed_data[2]
    confirmed_unit_quantities = confirmed_data[3]