Why nlohmann does not release memory

Question

I am using nlohmann library to parse JSON codes. We have two problems:
1- Why nlohmann use huge memory to parse data
2- After call parser in a function locally like below code it does not release memory. My JSON data size is about 8MB and the parser use more then 50MB for parsing. I parsed this JSON data 10 times and memory usage goes up to 600MB and after the function is finished memory did not released.

#include "nlohmann/json.hpp"

    #define REPEAT 10

        void GenerateNlohmann() {
          std::string filePath{FILE_ADDRESS};
          std::ifstream iFile(filePath.c_str(), std::ios::in);
          std::string data{};
          if (iFile.is_open()) {
            data = std::string((std::istreambuf_iterator<char>(iFile)),
                               std::istreambuf_iterator<char>()); // About 8MB size
            iFile.close();
          }
          if (!data.empty()) {
            nlohmann::json json = nlohmann::json::parse(data); // Use memory about 50MB
            std::vector<nlohmann::json> jsons{};
            for (int i = 0; i < REPEAT; ++i) {
              nlohmann::json j = nlohmann::json::parse(data);
              jsons.emplace_back(j);
            }
            while (!jsons.empty()) {
              jsons.pop_back();
            }
          }
        }

            int main() {
              GenerateNlohmann();

        // Now memory usage is about 600MB
              std::cout << "Input a numberto exit" << std::endl;
              int i;
              std::cin >> i;

              return 0;
            }

Answer 1

I had reproduce successful in ubuntu. My json data size about 1.4 KB but I parsed this json data for many times. Here are my test result:

Before Run:

KiB Mem :  8167476 total,  5461204 free,   284120 used,  2422152 buff/cache

1000 times:
KiB Mem :  8167476 total,  5456600 free,   288724 used,  2422152 buff/cache

10000 times:
KiB Mem :  8167476 total,  5405916 free,   339376 used,  2422184 buff/cache

100000 times:
KiB Mem :  8167476 total,  4893176 free,   852104 used,  2422196 buff/cache

After input the int (After run)
KiB Mem :  8167476 total,  5462208 free,   283116 used,  2422152 buff/cache

There is indeed a problem, but this is an optimization by the allocator (probably glibc in my case) and unrelated to the library.

If I add malloc_trim(0) in my code:

while (!jsons.empty()) {
      jsons.pop_back();
    }

+   malloc_trim(0);
  }

I will find everythink will be OK. In windows we can not reproduce because what we use is not glibc, I think.

Other Test: I had wrote other program to malloc many small memory with glibc, and the problem will still alive.My program is unrelated to the library, it just malloc and free many small memory.

Anyway, the probleam is unrelated to the library. If we add malloc_trim(0) into the library, there will be many calls during the parsering, which will reduce performance.So the better solution is add malloc_trim(0) in your code.