ValueError: pos_label=1 is not a valid label: array(['neg', 'pos'], dtype='<U3')

Question

I recieve this error while trying to obtain the recall score.

X_test = test_pos_vec + test_neg_vec
Y_test = ["pos"] * len(test_pos_vec) + ["neg"] * len(test_neg_vec)

recall_average = recall_score(Y_test, y_predict, average="binary")

print(recall_average)

This will give me:

    C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\metrics\classification.py:1030: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  if pos_label not in present_labels:
Traceback (most recent call last):
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 812, in <module>
    main()
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 91, in main
    evaluate_model(model, train_pos_vec, train_neg_vec, test_pos_vec, test_neg_vec, False)
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 648, in evaluate_model
    recall_average = recall_score(Y_test, y_predict, average="binary")
  File "C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\metrics\classification.py", line 1359, in recall_score
    sample_weight=sample_weight)
  File "C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\metrics\classification.py", line 1036, in precision_recall_fscore_support
    (pos_label, present_labels))
ValueError: pos_label=1 is not a valid label: array(['neg', 'pos'],
      dtype='<U3')

I tried to transform 'pos' in 1 and 'neg' in 0 this way:

for i in range(len(Y_test)):
     if 'neg' in Y_test[i]:
         Y_test[i] = 0
     else:
         Y_test[i] = 1

But this is giving me another error:

    C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\metrics\classification.py:181: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  score = y_true == y_pred
Traceback (most recent call last):
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 812, in <module>
    main()
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 91, in main
    evaluate_model(model, train_pos_vec, train_neg_vec, test_pos_vec, test_neg_vec, False)
  File "G:/PyCharmProjects/NB/accuracy/script.py", line 648, in evaluate_model
    recall_average = recall_score(Y_test, y_predict, average="binary")
  File "C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\metrics\classification.py", line 1359, in recall_score
    sample_weight=sample_weight)
  File "C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\metrics\classification.py", line 1026, in precision_recall_fscore_support
    present_labels = unique_labels(y_true, y_pred)
  File "C:\Users\anca_elena.moisa\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\utils\multiclass.py", line 103, in unique_labels
    raise ValueError("Mix of label input types (string and number)")
ValueError: Mix of label input types (string and number)

What I am trying to do is to obtain the metrics: accuracy, precision, recall, f_measure. With average='weighted', I obtain the same result: accuracy=recall. I guess this is not correct, so I changed the average='binary', but I have those errors. Any ideas?

Answer 1

recall_average = recall_score(Y_test, y_predict, average="binary", pos_label="neg")

Use "neg" or "pos" as pos_label and this error won't raise again.