I'm trying to fit a nonlinear least squares problem with BFGS (and L-BFGS-B) using optim. When I supply the analytical gradients, the linesearch terminates abnormally, and the final solution is always very close to the starting point. However, when I don't supply the gradients, it seems to converge fine. Does this suggest any numerical problems to anybody? I'm pretty sure the gradients are correct. Could it be a scaling issue? Thanks for any help.
You're pretty sure the gradients are right. Have you proved it? Have you calculated the gradients by finite difference and seen if they are about the same as the analytic gradients? That's the first place to look, I would think. I've had to do the same thing.
P.S. Have you considered Metropolis-Hastings? It's slow but robust, and no need for gradients or the Hessian.