Wrong statement for tokens remove

Question

I have created a token and I want to remove specific feature.

I use this:

toks <- tokens_remove(toks,
           remove_punct = TRUE, 
           remove_numbers = TRUE, 
           remove_symbols = TRUE)

However this is the error:

Error in tokens_select(x, ..., selection = "remove") : 
  unused arguments (remove_punct = TRUE, remove_numbers = TRUE, remove_symbols = TRUE)

How is the right syntax?

Answer 1

Using remove_xxx should be used when creating a tokens object with tokens not when using tokens_remove.

If you want to use tokens_remove with removing other items you could do something like this:

toks <- tokens_remove(tokens(txt, # a character, corpus, or tokens object 
                             remove_punct = TRUE, 
                             remove_numbers = TRUE, 
                             remove_symbols = TRUE
                             )
                      )