Is GridSearchCV computing SVC with rbf kernel and different degrees?

Question

I'm running a GridSearchCV with a OneVsRestClasssifer using SVC as an estimator. This is the aspect of my Pipeline and GridSearchCV parameters:

pipeline = Pipeline([
    ('clf', OneVsRestClassifier(SVC(verbose=True), n_jobs=1)),
    ])

parameters = {
    "clf__estimator__C": [0.1, 1],
    "clf__estimator__kernel": ['poly', 'rbf'],
    "clf__estimator__degree": [2, 3],
}

grid_search_tune = GridSearchCV(pipeline, parameters, cv=2, n_jobs=8, verbose=10)
grid_search_tune.fit(train_x, train_y)

According to the documentation of SVC the degree parameter is only used by the poly kernel:

http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html

degree : int, optional (default=3)

Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.

but when I see the output of my GridSearchCV it seems it's computing a different run for each SVC configuration with a rbf kernel and different values for the degree parameter.

[CV] clf__estimator__kernel=poly, clf__estimator__C=0.1, clf__estimator__degree=2
[CV] clf__estimator__kernel=poly, clf__estimator__C=0.1, clf__estimator__degree=2
[CV] clf__estimator__kernel=rbf, clf__estimator__C=0.1, clf__estimator__degree=2
[CV] clf__estimator__kernel=rbf, clf__estimator__C=0.1, clf__estimator__degree=2
[CV] clf__estimator__kernel=poly, clf__estimator__C=0.1, clf__estimator__degree=3
[CV] clf__estimator__kernel=poly, clf__estimator__C=0.1, clf__estimator__degree=3
[CV] clf__estimator__kernel=rbf, clf__estimator__C=0.1, clf__estimator__degree=3
[CV] clf__estimator__kernel=rbf, clf__estimator__C=0.1, clf__estimator__degree=3

Shouldn't all values of degree be ignored, when the kernel is set to rbf?

Answer 1

The output shown here is only the different combinations of parameters passed by the GridSearchCV to the internal estimator i.e. the SVC. But whether or not they are used depends upon the SVC. In this case, the SVC doesnt throw any error, but also doesnt use the degree. You should print the scores of all those combinations of which you are doubtful. They should be equal. That will tell you that the degree parameter is unused.

Note: Make sure to set the random_state of the GridSearchCV to duplicate the tests.

Explanation: The work of GridSearchCV is to just pass the parameters, the train data to estimator for fitting, then use the test data for scoring, and results the parameter combinations which resulted in best score.

When an incompatible combination of parameter is passed to an estimator, it is dependent on the implementation, whether the parameters are ignored or an error is raised for it.

For eg. in LogisticRegression, there are two parameters:

penalty : str, ‘l1’ or ‘l2’, default: ‘l2’ 
         Used to specify the norm used in the penalization.

solver : {‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’}, default: ‘liblinear’. 
         Algorithm to use in the optimization problem.
        ‘newton-cg’, ‘lbfgs’ and ‘sag’ only handle L2 penalty. 

As you can see, if I use l1 penalty with newton-cg solver, it results in incompatibility. So the estimator may chose to ignore the penalty paramter altogether or throw an error. In this case it throws an error.