Search code examples
statistics-bootstrap

boot.ci() can't compute confidence intervals for F1 score


I am trying to compute confidence intervals for F1 score using the boot package in R. But it doesn't seem to work and I am not sure why.

library(MLmetrics)
fci<-function(data,indices,x,y){
   d<-data[indices,]
   r<-F1_Score(x,y)
   r
}
set.seed(12345)
bootout<-boot(data=test,
              x=test$jbz,
              y=test$`DR_ai+jd`,
              R=2000,
              statistic=fci
              )
boot.ci(bootout)

Error:

[1] "All values of t are equal to  0.996637873066777 \n Cannot calculate confidence intervals"
NULL

Solution

  • library(MLmetrics)
    library(boot)
    set.seed(12345)
    fci<-function(data,indices,x,y){
       d<-as.data.frame(data[indices,])
       r<-F1_Score(d[,1],d[,2],positive=1)
       r
    }
    
    bootout<-boot(data=test,
                  x=test$jbz,
                  y=test$`DR_ai+jd`,
                  R=2000,
                  statistic=fci
                  )
    boot.ci(bootout,type="basic")
    F1_Score(y_true = test$jbz, y_pred = test$`DR_ai+jd`,positive="1")
    

    this works well~~