Search code examples
rmathematical-optimization

Is there any way to extract parameters and objective function for each iteration in R optimx

Suppose I have an optimization problem to solve using R, optimx. Is there any way I can extract the parameters and objective function values over time? 

f<-function(x){
  return(sum(abs(x)))
}

gr<-function(x){
  return(sign(x))
}
opt=optimx::optimx(runif(2),f,gr,method="BFGS")

The goal is trying to make such plot:

enter image description here

I think we can manually do it with Gradient Decent with following code, but how I can I do it in optimx?

x=c(0.5,1.5)
alpha=0.1
max_iter=20
x_trace=matrix(rep(0,max_iter*2),ncol=2)

for (i in 1:max_iter){
  x=x-alpha*gr(x)
  x_trace[i,]=x
}
f_trace=apply(x_trace,1,f)
Solution

  • Create a side effect:

    f<-function(x){
  .GlobalEnv$i <- get("i", envir = .GlobalEnv) + 1
  .GlobalEnv$log[get("i", envir = .GlobalEnv),] <- x
  return(sum(abs(x)))
}

gr<-function(x){
  return(sign(x))
}

library(optimx)
i <- 0
log <- matrix(numeric(100 * 2), ncol = 2)

opt <- optimx(c(0.8, -0.9),f,gr,method="BFGS")
log <- log[seq_len(i), ]

plot(log, type = "l", xlim = c(-2, 2), ylim = c(-1.2, 1.2))

    resulting plot

    Note that this includes all function calls, even those where the algorithm rejects the result and retries. control = list(trace = TRUE, REPORT = 1) lets optimx print the function values for accepted tries and you could capture.output this and use it to get only the parameters of these from log.

    It would be better to change optimx to return all accepted attempts, but I'm not going to invest that kind of effort. You could ask Prof. Nash if he would be willing to do this, but if you don't have a compelling common use case, he probably is not going to either.