I'm fitting a logistic regression model and am setting the random state to a fixed value.
Every time I do a "fit" I get different coefficients, example:
classifier_instance.fit(train_examples_features, train_examples_labels)
LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, penalty='l2', random_state=1, tol=0.0001)
>>> classifier_instance.raw_coef_
array([[ 0.071101940040772596 , 0.05143724979709707323, 0.071101940040772596 , -0.04089477198935181912, -0.0407380696457252528 , 0.03622160087086594843, 0.01055345545606742319,
0.01071861708285645406, -0.36248634699444892693, -0.06159019047096317423, 0.02370064668025737009, 0.02370064668025737009, -0.03159781822495803805, 0.11221150783553821006,
0.02728295348681779309, 0.071101940040772596 , 0.071101940040772596 , 0. , 0.10882033432637286396, 0.64630314505709030026, 0.09617956519989406816,
0.0604133873444507169 , 0. , 0.04111685986987245051, 0. , 0. , 0.18312324521915510078, 0.071101940040772596 ,
0.071101940040772596 , 0. , -0.59561802045324663268, -0.61490898457874587635, 1.07812569991461248975, 0.071101940040772596 ]])
classifier_instance.fit(train_examples_features, train_examples_labels)
LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, penalty='l2', random_state=1, tol=0.0001)
>>> classifier_instance.raw_coef_
array([[ 0.07110193825129411394, 0.05143724970282205489, 0.07110193825129411394, -0.04089477178162870957, -0.04073806899140903354, 0.03622160048165772028, 0.010553455400928528 ,
0.01071860364222424096, -0.36248635488413910588, -0.06159021545062405567, 0.02370064608376460866, 0.02370064608376460866, -0.03159783710841745225, 0.11221149816037970237,
0.02728295411479400578, 0.07110193825129411394, 0.07110193825129411394, 0. , 0.10882033461822394893, 0.64630314701686075729, 0.09617956493834901865,
0.06041338563697066372, 0. , 0.04111676713793514099, 0. , 0. , 0.18312324401049043243, 0.07110193825129411394,
0.07110193825129411394, 0. , -0.59561803345113684127, -0.61490899867901249731, 1.07812569539027203191, 0.07110193825129411394]])
I'm using version 0.14, the docs specify "The underlying C implementation uses a random number generator to select features when fitting the model. It is thus not uncommon, to have slightly different results for the same input data. If that happens, try with a smaller tol parameter."
I thought that setting the random state would make sure there is no randomness but apparently this is not the case. Is this a bug or desired behavior?
It's not really desired, but it's a known issue that is very hard to fix. The thing is that LogisticRegression
models are trained with Liblinear, which does not allow setting its random seed in a completely robust way. When you explicitly set the random_state
, a best effort is made to set Liblinear's random seed, but that may fail.