Search code examples
pythonoptimizationscipyscipy-optimize

How to add Tikhonov regularization in "scipy.optimize.least_squares"?


As the title, could anyone tell me a way to add "Tikhonov regularization" in "scipy.optimize.least_squares"? The default seems L2.


Solution

  • A straightforward way is to add more rows to your design matrix and objective vector.

    In other words if you have A and b, and seek x to minimise

    ||A*x-b||
    

    and you change A and b to

    A~ = (A)
         (K)
    b~ = (b)
         (k)
    

    then the least squares problem: minimise

    ||A~*x - b~||
    

    is minimising

    ||A*x-b||^2 + ||K*x-k||^2