Search code examples
pythonmachine-learningscikit-learnnlp

How to set custom stop words for sklearn CountVectorizer?

I'm trying to run LDA (Latent Dirichlet Allocation) on a non-English text dataset.

From sklearn's tutorial, there's this part where you count term frequency of the words to feed into the LDA:

tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2,
                            max_features=n_features,
                            stop_words='english')

Which has built-in stop words feature which is only available for English I think. How could I use my own stop words list for this?

Solution

  • You may just assign a list of your own words to the stop_words, e.g.:

    stop_words = (["word1", "word2","word3"])