Search code examples
c++performanceramlarge-datachaiscript

How does chaiscript handle big objects?


I am considering using chaiscript for my project. However, I have a question about performance. Maybe it has been answered already somewhere, but I couldn't find it...

I have a simulation using large data structures (at least 1-2GB). Therefore, I fear that I will blow my RAM by doing something like this in chaiscript:

var data = create_big_structure();
for (var i = 1; i < max; ++i)
{
   var new_data = update_structure(i, data);
   output_data_evolution(data, new_data);
   data = new_data;
}
//data is not needed anymore

My questions are:

  1. Will chaiscript delete the data between each loop execution? Namely new_data...
  2. Will chaiscript delete the data after exiting loops? Again new_data...
  3. If the answer to 1. and 2. is yes, is there another way I would need to check, to still be safe?
  4. Will chaiscript delete unused variables? Namely data, after the loop... (I guess the answer is no.)

Thanks for the help!


Solution

  • After lots of testing, I found the answer to my questions with the following code:

    #include <vector>
    #include <chaiscript/chaiscript.hpp>
    
    std::vector<int> create_big_structure() {
      //This is 1GB in size.
      return std::vector<int>(268435456);
    }
    std::vector<int> update_structure(int i, const std::vector<int>& data) {
      //This is 1GB in size.
      return std::vector<int>(268435456);
    }
    void output_data_evolution(const std::vector<int>& data, const std::vector<int>& new_data) {}
    
    int main() {
      chaiscript::ChaiScript chai;
    
      chai.add(chaiscript::fun(&create_big_structure), "create_big_structure");
      chai.add(chaiscript::fun(&update_structure), "update_structure");
      chai.add(chaiscript::fun(&output_data_evolution), "output_data_evolution");
      chai.add(chaiscript::bootstrap::standard_library::vector_type<std::vector<int>>("VectorInt"));
      chai.eval(R"(
        var max = 5;
        var data = create_big_structure();
        for (var i = 1; i < max; ++i)
        {
          var new_data = update_structure(i, data);
          output_data_evolution(data, new_data);
          data = new_data;
        }
      )");
    }
    

    I ran the code using MSVC, and looked at the runtime statistics to figure out what happened in the RAM: Runtime Statistics

    The code works reasonably. After a startup phase, 1GB of RAM is allocated for the data object. In the loop, the RAM stays at 2GB, because we also have the new_data object. After the loop, it falls down to 1GB.

    Therefore, the answers to my questions are:

    1. chaiscript does indeed delete the data after each looping of the for statement.
    2. chaiscript does delete the data after the loops as well.
    3. Yes, you need to check that the c++ functions do not create unnecessary data. For example, if you write with a copy instead of a reference -- update_structure(int i, std::vector<int> data), then the function will use a copy of data, and therefore the RAM will jump to 3GB in the loop.
    4. chaiscript does not delete unused data inside the same block. (new_data is deleted after the loop, but not data.)