Search code examples
jsonmongodbperloptimizationhashref

Save json string to mongodb in perl efficiently (without converting to hash-ref)

I would like to insert data to mongodb in perl. I can insert perl objects like hash-ref. But I want to append to them also prepared JSONs.

I have these JSONs in text files and I can transform them to hash-ref and then put to database but I looking for more efficient way because of amount of data, that I need to process.

It is possible? I can do inserts but I looking for optimization.

Similar topic (but without answer for this question):

Insert into mongodb with perl

Technical aspects:

For one insert there are processed one file 100kB - 1MB I contains 4 JSON strings among rest of text, any string about 2 - 15 k characters. I getting some properties from file and rest of text and has it in hash-ref. I do not want any information from this JSON in rest of my program. I am interested only put them together into database.

Solution

  • There's no direct way to insert JSON into MongoDB. It always has to be processed into MongoDB's wire format. For Perl that means JSON decoding and then inserting with the driver, which, as you point out, has overhead.

    If you just have JSON data, the best thing might be to use the mongoimport tool that comes with the database.