In the documentations for scipy.optimize.root with method = lm the following are the default for options keyword.
options={'col_deriv': 0, 'diag': None, 'factor': 100, 'gtol': 0.0, 'eps': 0.0, 'func': None, 'maxiter': 0, 'xtol': 1.49012e-08, 'ftol': 1.49012e-08}
On the description for col_deriv they say that
col_deriv : bool, optional
non-zero to specify that the Jacobian function computes derivatives down the columns (faster, because there is no transpose operation)
If I understand the statement it says that if I write col_deriv = True for example, the jacobian will be computed column wise and therefore faster.
Question: If it is faster, why isn't a non zero value for col_deriv the default value?.
Am I missing something here?.
Maybe the documentation from scipy.optimize.leastsq could help since it documents both Dfun (Jacobian) and col_deriv. From Dfun we get:
Dfun : callable, optional
A function or method to compute the Jacobian of func with derivatives across the rows. If this is None, the Jacobian will be estimated.
From col_deriv we get:
col_deriv : bool, optional
non-zero to specify that the Jacobian function computes derivatives down the columns (faster, because there is no transpose operation).
My reading of this is as follows:
By default, scipy expects that a function that computes the Jacobian matrix return a matrix that follows "normal" definition (see, e.g., https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant).
However, scipy itself calls other functions, possibly written in Fortran see, e.g., minpack, which expect that derivatives (with regard to coordinates) are placed in columns.
It follows that if the function that computes the Jacobian matrix could return a matrix with derivatives placed along columns instead of rows, then scipy will not need to transpose the Jacobian matrix before passing it to the minpack function thus saving computational time.