Reading Large JSON file into variable in C#.net

Question

I am trying to parse the JSON files and insert into the SQL DB.My parser worked perfectly fine as long as the files are small (less than 5 MB).

I am getting "Out of memory exception" when trying to read the large(> 5MB) files.

if (System.IO.Directory.Exists(jsonFilePath))
                {
                    string[] files = System.IO.Directory.GetFiles(jsonFilePath);
                    foreach (string s in files)
                    {
                        var jsonString = File.ReadAllText(s);
                        fileName = System.IO.Path.GetFileName(s);
                        ParseJSON(jsonString, fileName);

                    }
                }

I tried the JSONReader approach, but no luck on getting the entire JSON into string or variable.Please advise.

Answer 1

Use 64 bit, check RredCat's answer on a similar question:

Newtonsoft.Json - Out of memory exception while deserializing big object

NewtonSoft Jason Performance Tips

Read the article by David Cox about tokenizing:

"The basic approach is to use a JsonTextReader object, which is part of the Json.NET library. A JsonTextReader reads a JSON file one token at a time. It, therefore, avoids the overhead of reading the entire file into a string. As tokens are read from the file, objects are created and pushed onto and off of a stack. When the end of the file is reached, the top of the stack contains one object — the top of a very big tree of objects corresponding to the objects in the original JSON file"

Parsing Big Records with Json.NET