Search code examples
rmathematical-optimizationmodel-fitting

How does one supply a gradient (or hessian) to nlm?


In the helpfile for the non linear minimiser function stats::nlm it states that one of the arguments f is (emphasis mine):

the function to be minimized, returning a single numeric value. This should be a function with first argument a vector of the length of p followed by any other arguments specified by the ... argument. If the function value has an attribute called gradient or both gradient and hessian attributes, these will be used in the calculation of updated parameter values.

Does this mean that gradients and hessians are supplied by a command like:

attr(f, 'gradient') <- function(...){...}

What should the input and outputs of this function be?

Contrast this to the general optimiser stats::optim:

optim(par, fn, gr = NULL, ...,
  method = c("Nelder-Mead", "BFGS", "CG", "L-BFGS-B", "SANN",
             "Brent"), ...)

where the gradient is explicitly specified as an argument to optim, as an example of this see: how to propery specify a gradient function for use in optim() or other optimizer

Examining the code internals of nlm did not help.


Solution

  • Here's a simple one-dimensional example:

    f <- function(x) {
      out <- -exp(-0.5 * x^2)
      attr(out, 'gradient') <- -x * out 
      attr(out, 'hessian') <-  (x^2 - 1) * out
      return(out)
    }
    
    nlm(f, 1.3, hessian = TRUE, check.analyticals = TRUE)
    

    Which gives:

    # $minimum
    # [1] -1
    # 
    # $estimate
    # [1] 4.23687e-14
    # 
    # $gradient
    # [1] 4.23687e-14
    # 
    # $hessian
    # [,1]
    # [1,]    1
    # 
    # $code
    # [1] 1
    # 
    # $iterations
    # [1] 3