Search code examples
pythonnumpyscipymathematical-optimizationleast-squares

How to include data weight to Scipy NNLS function?

I want to use the scipy nnls function, however, this does not support adding data weighting. 

import numpy as np
from scipy.optimize import nnls 

A = np.array([[60, 90, 120], 
              [30, 120, 90]])

b = np.array([67.5, 60])


W = np.array([2, 3])

So I am wondering how can I add this W matrix into non-nagative least square?

Solution

  • So if I understand the question correctly, you're trying to minimize Σᵢ Wᵢ(Ax − b)ᵢ², constrained by xᵢ ≥ 0 for all i, rather than just Σᵢ(Ax − b)ᵢ². This is then equivalent to minimizing Σᵢ (diag(sqrt(W))Ax − diag(sqrt(W))b)ᵢ², where sqrt(W) denotes the element-wise square root of W. This you can solve with scipy.optimize.nnls directly.

    Note that in your example, this becomes rather boring since it is possible to find an x with positive entries and with Ax = b, meaning that the weighted case becomes trivial as well.

    An example where this makes an actual difference, consider the following, where we put most of the weight on the first coordinate:

    In [48]: A
Out[48]:
array([[ -40,   90, -120],
       [  30,  120,  -90]])

In [49]: b
Out[49]: array([67.5, 60. ])

In [50]: W = np.array([1000, 3])

In [51]: A @ nnls(A, b)[0]
Out[51]: array([53.1, 70.8])

In [52]: A @ nnls(np.sqrt(W)[:, None] * A, np.sqrt(W) * b)[0]
Out[52]: array([67.3806366 , 89.84084881])