I need to implement Naive Bayes classifier and plot ROC curve for
1) only 2 classes(Trousers and pullovers) out of 10 classes of the [FMNIST][1] dataset and then
2) second for all the ten classes without using scikit library and just basic matplotlib, pandas libraries from scratch.
I have successfully implemented the Naive bayes classifier for both the cases but I am not able to understand how to implement ROC curve since it requires to set the threshold. Naive bayes classifier is just using the likelihood and prior to get posteriors for discrete distributions and the feature values are RGB(0-255) values which are binarized using threshold of 127 (number of features per test sample is 28*28= 784). For 10 classes, I think I need to plot 10 curves by taking one of the classes as positive and rest negative one by one.
My classifier predicts the class for a test sample based on the maximum of the posterior probabilities for all the classes. But I am unable to understand how I can decide the threshold and how to plot ROC Curve. I went through some of the stackoverflow and other links but couldn't get the understanding. Please explain as I am new to machine learning.
Bit late but...
In the binary case, rather than take the prediction to be the class with the largest posterior, just keep the posterior for the affirmative case and compare that with your threshold.
For example, if your threshold was 0.8, and 1 and 0 denote positive and negative, respectively; then your prediction would be 1 if P(Y=1|X) >= 0.8 and 0 otherwise.