Search code examples
rmemoryrservepyrserve

Memory management with Rserve on Windows

I need to perform various tasks with Python: sensitivity analysis, optimization and simulation launching. However the first postprocessing of simulation results relies on R and a bulky database stored in Rdata (8 Gb with some data tables representing up to 3 Gb). So I need to iterate the call of R scripts without iterating the data loading (which takes several minutes). I first tried with pyRserve but I am facing memory limits that seem more restrictive than with raw R.

With:

import pyRserve
conn = pyRserve.connect()
conn.r.load('.Rdata')

I get the following error:

REvalError: Error: cannot allocate vector of size 640.3 Mb

So:

1/ Is there a way to allocate more memory to Rserve (typically the amount allocated in raw R i.e. 16 Gb with my configuration)?

2/ Is there an alternative to Rserve in order to load Rdata once and for all and then run R scripts within the same workspace?

Solution

  • The following works perfectly.

    memory_limit = getattr(conn.r, 'memory.limit')
memory_limit(16287.0)