Search code examples
rtext-miningtmword-cloudmetacharacters

text mining with tm package in R ,remove words starting from [http] or any other specifc word

I am new to R and text mining. I had made a word cloud out of twitter feed related to some term. The problem that I'm facing is that in the wordcloud it shows http:... or htt... How do I deal about this issue I tried using metacharacter * but I still doubt if I'm applying it right

tw.text = removeWords(tw.text,c(stopwords("en"),"rt","http\\*"))

somebody into text-minning please help me with this.

Solution

  • If you are looking to remove URLs from your string, you may use:

    gsub("(f|ht)tp(s?)://(.*)[.][a-z]+", "", x)

    Where x would be:

    x <- c("some text http://idontwantthis.com", 
         "same problem again http://pleaseremoveme.com")

    It would be easier to provide you with a specific answer if you could post sample of your data but the following example would give you a clean text with no URLs:

    > clean_x <- gsub("(f|ht)tp(s?)://(.*)[.][a-z]+", "", x)
> clean_x
[1] "some text "          "same problem again "

    As a side point, I would suggest that it may be worth searching for the existing methods to clean text before mining. For example the clean function discussed here would enable you to do this automatically. On similar lines, there are function to clean your text from tweets (#,@), punctuation and other undesirable entries.