Apply NLTK stemming on pandas column/index

Question

I want to stem the columns and index of a DataFrame to something like this

ps = PorterStemmer()
df_dic = pd.read_csv('inquirerbasic_clean.csv', sep=';', index_col=0).T
print(type(df_dic))  # pandas.core.frame.DataFrame
df_dic.index = ps.stem(df_dic.index.str.lower())
df_dic.columns = ps.stem(df_dic.columns.str.lower())

and I get this error:

  File "<ipython-input-18-0156717e5956>", line 5, in <module>
    df_dic.index = ps.stem(df_dic.index.str.lower())

  File "/usr/lib/python3.6/site-packages/nltk/stem/porter.py", line 632, in stem
    stem = self.stem_word(word.lower(), 0, len(word) - 1)

AttributeError: 'Index' object has no attribute 'lower'

Also, if I convert the index to a list:

ps.stem(list(df_dic.index.str.lower()))

I get an equivalent error message:

  File "/usr/lib/python3.6/site-packages/nltk/stem/porter.py", line 632, in stem
    stem = self.stem_word(word.lower(), 0, len(word) - 1)

AttributeError: 'list' object has no attribute 'lower'

So, how can I stem them?

Answer 1

These work on strings, not lists, so apply ps.stem using map.

df_dic.index = df_dic.index.str.lower().map(ps.stem)
df_dic.columns = df_dic.columns.str.lower().map(ps.stem)

If that doesn't sit well with you (for whatever reason), use a list comprehension:

df_dic.index = [ps.stem(v.lower()) for v in df_dic.index]

And so on.