Search code examples
pythonlasso-regressioncvxpy

ECOS solver stops working in CVXPY 1.0 version

I was working with cvxpy 0.4 version, and in this version I had programmed the group lasso penalized linear model as follows:

from cvxpy import *
from sklearn.datasets import load_boston
import numpy as np
boston = load_boston()
x = boston.data
y = boston.target
index = np.array([1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5])
lambda_val = 1

0.4 version

n = x.shape[0]
lambda_param = Parameter(sign="positive")
index = np.append(0, index)
x = np.c_[np.ones(n), x]
group_sizes = []
beta_var = []
unique_index = np.unique(index)
for idx in unique_index:
    group_sizes.append(len(np.where(index == idx)[0]))
    beta_var.append(Variable(len(np.where(index == idx)[0])))
num_groups = len(group_sizes)
group_lasso_penalization = 0
model_prediction = x[:, np.where(index == unique_index[0])[0]] * beta_var[0]
for i in range(1, num_groups):
    model_prediction += x[:, np.where(index == unique_index[i])[0]] * beta_var[i]
    group_lasso_penalization += sqrt(group_sizes[i]) * norm(beta_var[i], 2)
lm_penalization = (1.0 / n) * sum_squares(y - model_prediction)
objective = Minimize(lm_penalization + (lambda_param * group_lasso_penalization))
problem = Problem(objective)
lambda_param.value = lambda_val
problem.solve(solver=ECOS)
beta_sol = [b.value for b in beta_var]

1.0 Version

n = x.shape[0]
lambda_param = Parameter(nonneg=True)
index = np.append(0, index)
x = np.c_[np.ones(n), x]
group_sizes = []
beta_var = []
unique_index = np.unique(index)
for idx in unique_index:
    group_sizes.append(len(np.where(index == idx)[0]))
    beta_var.append(Variable(shape=(len(np.where(index == idx)[0]), 1)))
num_groups = len(group_sizes)
model_prediction = 0
group_lasso_penalization = 0
model_prediction = x[:, np.where(index == unique_index[0])[0]] * beta_var[0]
for i in range(1, num_groups):
    model_prediction += x[:, np.where(index == unique_index[i])[0]] * beta_var[i]
    group_lasso_penalization += sqrt(group_sizes[i]) * norm(beta_var[i], 2)
lm_penalization = (1.0 / n) * sum_squares(y.reshape((n, 1)) - model_prediction)
objective = Minimize(lm_penalization + (lambda_param * group_lasso_penalization))
problem = Problem(objective)
lambda_param.value = lambda_val
problem.solve(solver=ECOS)
beta_sol = [b.value for b in beta_var]

While using the 1.0 code version, it shows this error message: image

So, I think I have migrated correctly the code from version 0.4 to 1.0 but a problem that in 0.4 version was solved with ECOS solver, in 1.0 version shows an error message. Am I doing something wrong here? Just in case it matters, I am runnig this code in miniconda python 2.7, in a windows machine.

Solution

  • It appears your code isn't consistent with respect to the beta_vars. In the 0.4 example they are one dimensional, but in 1.0 example they are 2 dimensional. Here's a simple program that shows cvxpy is consistent:

    import cvxpy
print(cvxpy.__version__)
from cvxpy import *
try:
    x = Variable(shape=(3, 3))
except TypeError:
    x = Variable(3, 3)
obj = Minimize(norm(x, 2))
Problem(obj).solve(solver=ECOS)

    Output from 0.4:

    0.4.11
Traceback (most recent call last):
  File "./cvxtest.py", line 11, in <module>
    Problem(obj).solve(solver=ECOS)
  File "lib/python3.6/site-packages/cvxpy/problems/problem.py", line 209, in solve
    return self._solve(*args, **kwargs)
  File "lib/python3.6/site-packages/cvxpy/problems/problem.py", line 321, in _solve
    solver.validate_solver(constraints)
  File "lib/python3.6/site-packages/cvxpy/problems/solvers/solver.py", line 131, in validate_solver
    self._reject_problem("it cannot solve semidefinite problems")
  File "lib/python3.6/site-packages/cvxpy/problems/solvers/solver.py", line 156, in _reject_problem
    raise SolverError(message)
cvxpy.error.SolverError: The solver ECOS cannot solve the problem because it cannot solve semidefinite problems.

    Output from 1.0:

    1.0.8
Traceback (most recent call last):
  File "./cvxtest.py", line 11, in <module>
    Problem(obj).solve(solver=ECOS)
  File "lib/python3.6/site-packages/cvxpy/problems/problem.py", line 247, in solve
    return solve_func(self, *args, **kwargs)
  File "lib/python3.6/site-packages/cvxpy/problems/problem.py", line 357, in _solve
    raise e
  File "lib/python3.6/site-packages/cvxpy/problems/problem.py", line 355, in _solve
    solver=solver)
  File "lib/python3.6/site-packages/cvxpy/reductions/solvers/solving_chain.py", line 143, in construct_solving_chain
    ", ".join([cone.__name__ for cone in cones])))
cvxpy.error.SolverError: Either candidate conic solvers (['ECOS']) do not support the cones output by the problem (PSD), or there are not enough constraints in the problem.

    In summary you should fix your 1.0 code to make your Variables one dimensional.