I'm trying to fit a Bayesian model using brms::brm()
replacing the prior of one parameter in the model
one after another by purrr::map2()
(I have 63 priors for that parameter). I can 'theoretically' save the each fitted model as a distinct object in the global environment (i.e. my workspace) by list2env()
, thanks to the answer of my previous question. By list2env()
, the objects (brmsfit
s) would be saved after the all 63 iterations are done. However, when I run the whole code, always I get an error message saying Error in scan(con, nlines = 1, sep = ",", quiet = TRUE) : could not allocate memory (0 Mb) in C function 'R_AllocStringBuffer'
and no brmsfit
object is stored in the global environment, although the model fitting seems to be done 63 times, as much as my prior exist.
Therefore, I would like to save each brmsfit
object as an R object and as an .Rds
or .Rda
file immediately after its iteration done as to avoid such a memory problem. But, what should I do to realise that?
Note that the following command is just a "schematic" example, with publicly available data, of what I'm trying to do. It will work without the problem I mentioned above, since the data and the model in brm()
in this example are way simpler and the number of priors is much fewer than what I'm tackling.
library(lme4)
library(tidyverse)
library(magrittr)
library(brms)
library(cmdstanr)
library(rstan)
## Parallelize the chains using all the cores:
options(mc.cores = parallel::detectCores())
prior_test <- data.frame(
sd = c(0.01, 0.01, 0.01),
mean = c(50, -50, 0)
) %>%
mutate(
id = row_number()
)
list2env(
purrr::map2(
prior_test$sd,
prior_test$mean,
function(psd, pm){
gc()
gc()
cbfm1 <- brm(
Reaction ~ 0 + Intercept + Days + (0 + Intercept + Days|Subject),
data = sleepstudy,
family = "normal",
prior =
c(
prior(normal(0, 1), class = b, coef = Intercept),
set_prior(
paste0(
"normal(",
pm,
", ",
psd,
")"
),
class = "b",
coef = "Days"
),
prior(normal(0, 1), class = sd),
prior(lkj(2), class = cor)
),
save_pars = save_pars(all = TRUE),
backend = "cmdstanr"
)
}
) %>%
setNames(paste0('model', prior_test$id)),
.GlobalEnv
)
Error in scan(con, nlines = 1, sep = ",", quiet = TRUE) :
could not allocate memory (0 Mb) in C function 'R_AllocStringBuffer'
15.
scan(con, nlines = 1, sep = ",", quiet = TRUE)
14.
rstan::read_stan_csv(out$output_files())
13.
.fit_model(model, ...)
12.
.fun(model = .x1, sdata = .x2, algorithm = .x3, backend = .x4,
iter = .x5, warmup = .x6, thin = .x7, chains = .x8, cores = .x9,
threads = .x10, inits = .x11, exclude = .x12, control = .x13,
future = .x14, seed = .x15, silent = .x16)
11.
eval(expr, envir, ...)
10.
eval(expr, envir, ...)
9.
eval2(call, envir = args, enclos = parent.frame())
8.
do_call(fit_model, fit_args)
7.
brm(voice ~ 0 + Intercept + agent + patient + REGION1 + REGION2 +
agent:patient + agent:REGION1 + agent:REGION2 + patient:REGION1 +
patient:REGION2 + agent:patient:REGION1 + agent:patient:REGION2 +
(0 + Intercept + agent + patient + agent:patient | subj) + ...
6.
.f(.x[[i]], .y[[i]], ...)
5.
map2(R1data$prior_sd, R1data$prior_mean, function(psd, pm) {
gc()
gc()
brm(voice ~ 0 + Intercept + agent + patient + REGION1 + REGION2 + ...
4.
eval(lhs, parent, parent)
3.
eval(lhs, parent, parent)
2.
map2(R1data$prior_sd, R1data$prior_mean, function(psd, pm) {
gc()
gc()
brm(voice ~ 0 + Intercept + agent + patient + REGION1 + REGION2 + ...
1.
list2env(map2(R1data$prior_sd, R1data$prior_mean, function(psd,
pm) {
gc()
gc() ...
I think you could use something like the following (which is a simpler version that avoids where your data comes from, variable names, model specification, and all that)
dir.create("models")
# This function generates a file path, runs the `brm` function and saves the object
# returned by `brm()` as a .rds file
my_fun = function(prior_sd, prior_mean) {
file_name = paste0("models/model_sd_", prior_sd, "_mean_", prior_mean, ".rds")
brms_object = brm(...) # pass formula, prior, data, options, etc.
saveRDS(brms_object, file_name) # you can control compression too if you want
}
sds = c(0.01, 0.01, 0.01)
means = c(50, -50, 0)
purrr::walk2(sds, means, my_fun)
You have to make sure that the data frame you're going to use exists in the global environment