Search code examples
rnlptm

tm package removeWords function concatenate words in R


Am cleaning the sample data using removewords from tm package but removeWords function concatenate the words post removal. It should be "environmental dead frog" "environmental dead mouse". Can somebody guide ?

library(tm)
dc<-c("environmental dead frog still","environmental dead mouse come")

manualremovelist<-c("the","does","doesn't","please","new","ok","one","cant",
                "doesnt","can","still","done","will","without","seen",
                "also","danfoss","case","doesn´t","due","need","occurs","made",
                "using","now","make","makes","needs","put","okay","sno","since","therefore",
                "found","milwaukee","probably","got","finally","isnt","per","two",
                "obvious","unable","must","nos","3nos","1no",".","phone","tel","attached",
                "given","find","have","see","be","give","do","come","use","make","get",
                "try","call","request")

dc<-removeWords(dc,manualremovelist)

"environmentaldeadfrog"  "environmentaldeadmouse"

Solution

  • removeWords works only for words. You may split the string into words and use removeWords on individual phrases/sentences.

    library(tm)
    
    dc  <- sapply(strsplit(dc, '\\s+'), function(x) 
            trimws(paste0(removeWords(x, manualremovelist), collapse = ' ')))
    
    dc
    
    #[1] "environmental dead frog"  "environmental dead mouse"