any way to select univariate features based on wilcoxon test in R?

Question

I intend to use care::sbf to do univariate feature selection, wheres my input is dataframe with mulitple variables (a.k.a, its columns), list of candidate features, and label (a.k.a, categorical variables). After I read caret package documentation, I tried of using sbf, sbfController to do feature selection, but I ran into an error down below:

Error in contrasts<-(*tmp*, value = contr.funs[1 + isOF[nn]]) :
contrasts can be applied only to factors with 2 or more levels

can anyone point me how to resolve this error? what's correct of using caret::sbf to do feature selection? any thought?

reproducible example:

here is the reproducible example on public gist where I used it as input.

my current attempt:

library(caret)
library(e1071)
library(randomForest)

df=read.csv("df.csv", header=True)

sbfCtrl <- sbfControl(method = 'cv', number = 10, returnResamp = 'final', functions = caretFuncs, saveDetails = TRUE)

model <- sbf(form= ventil_status~ .,
                 data= df,
                 methods='knn',
                 trControl=trainControl(method = 'cv', classProbs = TRUE),
                 tuneGrid=data.frame(k=1:10),
                 sbfControl=sbfControl(functions = sbfCtrl,
                                       methods='repeatedcv', number = 10, repeats = 10))

print(model)
print(model$fit$results)

> model <- sbf(ventil_status~ ., data=df, sizes=c(1,5,10,20),
+              method= 'knn', trControl=trainControl(method = 'cv', classProbs = TRUE),
+              tuneGrid = data.frame(k=1:10),
+              sbfControl=sbfCtrl)
Error in `contrasts<-`(`*tmp*`, value = contr.funs[1 + isOF[nn]]) : 
  contrasts can be applied only to factors with 2 or more levels

I googled this error but still couldn't get over it. Any idea to make the above code work? what's the correct way to do filter selection by using caret::sbf ?

what I want is output dataframe must have selected features with its p-value attached to it. So here is my attempt:

newdf <- df[ , -which(names(df) %in% c("subject"))]
p_value_vector <- sapply(names(newdf), function(i) 
    tryCatch(
        wilcox.test(newdf[newdf$ventil_status %in% "0", i], 
                        newdf[newdf$ventil_status %in% "1", i], 
                    na.action(na.omit))$p.value),
    warning = function(w) return(NA),
    error = function (e) return(NA)
)

expected output:

I am expecting output dataframe with selected features wheres its p-value returned by wilcox.test should be attached to corresponding features. any idea to make this happen in r? How can I operate feature selection using caret::sbf properly? any thought?

here is my R sessioninfo:

> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] ggpubr_0.2.5        magrittr_1.5        reshape2_1.4.3     
 [4] forcats_0.5.0       purrr_0.3.3         readr_1.3.1        
 [7] tibble_2.1.3        tidyverse_1.3.0     stringr_1.4.0      
[10] dplyr_0.8.5         scales_1.1.0        tidyr_1.0.2        
[13] aws.s3_0.3.20       randomForest_4.6-14 e1071_1.7-3        
[16] mlbench_2.1-1       caret_6.0-86        ggplot2_3.3.0      
[19] lattice_0.20-38  

Answer 1

For using sbf, you can use caretSBF and then add in the score and filter as you like them defined:

library(mlbench)
library(caret)

knnSBF = caretSBF
knnSBF$summary <- twoClassSummary
knnSBF$score <- function(x, y) {
    wilcox.test(x ~ y)$p.value
}
knnSBF$filter <- function(score, x, y) {
     score <= 0.05
}

Then you define the training parameters and sbf parameters:

sbfCtrl <- sbfControl(method = "cv",number = 3,
functions = knnSBF,saveDetails = TRUE)

trn_grid <- expand.grid(k=c(2,6,10))

trCtrl <-  trainControl(method = "cv",number = 3,
                        classProbs = TRUE,verboseIter = TRUE)

Then run the train:

data(Sonar)
y = Sonar$Class
x = Sonar[,-ncol(Sonar)]
set.seed(111)
model1 <- sbf(x,y,trControl = trCtrl,
                sbfControl = sbfCtrl,
                method = "knn",
                tuneGrid = trn_grid)

model1$variables
$selectedVars
 [1] "V1"  "V2"  "V3"  "V4"  "V5"  "V6"  "V8"  "V9"  "V10" "V11" "V12" "V13"
[13] "V14" "V20" "V21" "V22" "V36" "V37" "V42" "V43" "V44" "V45" "V46" "V47"
[25] "V48" "V49" "V50" "V51" "V52" "V54" "V58"

$selectedVars
 [1] "V4"  "V5"  "V6"  "V9"  "V10" "V11" "V12" "V13" "V14" "V20" "V21" "V22"
[13] "V28" "V31" "V34" "V35" "V36" "V37" "V43" "V44" "V45" "V46" "V47" "V48"
[25] "V49" "V51" "V52"

$selectedVars
 [1] "V1"  "V2"  "V3"  "V4"  "V5"  "V6"  "V7"  "V8"  "V9"  "V10" "V11" "V12"
[13] "V13" "V14" "V21" "V22" "V23" "V34" "V35" "V36" "V37" "V43" "V44" "V45"
[25] "V46" "V47" "V48" "V49" "V50" "V51" "V52" "V53" "V56" "V58"

I don't think they return you the p-values, although I might be wrong. For you function to calculate the p-values, using the above example

p_value_vector <- apply(x,2,function(i)wilcox.test(i~y)$p.value)