Search code examples
rmathematical-optimizationnlopt

nloptr: does lbfgs() need to explicitly provide a gradient function when doing optimization?

In the nloptr package, functions like lbfgs() seem to need a gradient function. But if I do not provide the gradient function, they also work.

My question is: does nloptr automatically calculate the gradient function, or do functions like lbfgs() just not need the gradient function?

If the objective function is very complex, can nloptr calculate the gradient function automatically, or must it be provided by users? 

library(nloptr)

## example for auglag()
x0 <- c(1, 1)
fn <- function(x) {
  (x[1] - 2) ^ 2 + (x[2] - 1) ^ 2
}

hin <- function(x) {
  -0.25 * x[1] ^ 2 - x[2] ^ 2 + 1 # hin >= 0
}

heq <- function(x) {
  x[1] - 2 * x[2] + 1 # heq == 0
}
## it works even gr = NULL
auglag(x0, fn, gr = NULL, hin = hin, heq = heq, localsolver = "LBFGS")

gr <- function(x) nl.grad(x, fn)
## it also works, when provide the gradient function.
auglag(x0, fn, gr = gr, hin = hin, heq = heq, localsolver = "LBFGS")
Solution

  • From ?lbfgs, we read that this function does indeed automatically compute the gradient if it is not provided:

    gr: gradient of function fn; will be calculated numerically if not specified.

    Digging into the source code of lbfgs, we see that it does this with the nl.grad function:

    if (is.null(gr)) {
    gr <- function(x) nl.grad(x, fn)
}

    From the source code of nl.grad (or from ?nl.grad), it is clear that the function performs central differences to numerically compute the gradient. This approach to estimating the gradient merely evaluates a function of k variables at 2k nearby points, which should work both for simple and complex functions alike.