Trying to read BSON file, get bson.errors.InvalidBSON: objsize too large

Question

I am currently trying to read a bson file to import it into a database. I can already read the file and print it as as bytes, but I end up just getting an bson.errors.InvalidBSON: objsize too large error.

This is the code that is trying to decode the file

with zip.open(name) as myfile:
    content = myfile.read()
    print(content)
    print(bson.decode_all(content))

and this is the output i get

b'[{"_id": {"$oid": "5bf3cf511c9d44000088c376"}, "some": "sort of"}, {"_id": {"$oid": "5bf3cf5c1c9d44000088c377"}, "test": "data"}]'
Traceback (most recent call last):
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/bin/mongo-backup", line 11, in <module>
    load_entry_point('mongo-backup-cli', 'console_scripts', 'mongo-backup')()
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/jonas/work/mongodb-backup-py/mongo_backup/cli.py", line 45, in restore
    restore_files(uri, db_name, file.name)
  File "/home/jonas/work/mongodb-backup-py/mongo_backup/restore.py", line 6, in restore_files
    print(read_zip_file(file))
  File "/home/jonas/work/mongodb-backup-py/mongo_backup/zip.py", line 24, in read_zip_file
    print(bson.decode_all(content))
bson.errors.InvalidBSON: objsize too large

Answer 1

print(content)

b'[{"_id": {"$oid": "5bf3cf511c9d44000088c376"}, "some": "sort of"}, {"_id": {"$oid": "5bf3cf5c1c9d44000088c377"}, "test": "data"}]'

the bytes in the content variable is json encoded bson, not plain bson

if this is the output format you intend to continue to use, you need to change your code to use bson's JSON utility to load the string into a python object:

with zip.open(name) as myfile:
    content = myfile.read()
    print(content)
    print(bson.json_util.loads(content))
    #         ^--------------^
    #                | this stuff