Count punctuation in a DataFrame column

Question

I'm trying to count punctuation in the content column of this DataFrame. I've tried this and it doesn't work. My DataFrame looks like this:

I want the result to be like this: but instead of emotions to count the punctuation of every article.

In:

text_words = df.content.str.split()
punctuation_count = {}
punctuation_count[','] = 0
punctuation_count[';'] = 0
punctuation_count["'"] = 0
punctuation_count['-'] = 0

def search_for_single_quotes(word):
    single_quote = "'"
    search_char_index = word.find(single_quote)
    search_char_count = word.count(single_quote)
    if search_char_index == -1 and search_char_count != 1:
        return
    index_before = search_char_index - 1
    index_after = search_char_index + 1

    if index_before >= 0 and word[index_before].isalpha() and index_after == len(word) - 1 and word[index_after].isalpha():
        punctuation_count[single_quote] += 1

for word in text_words:
    for search_char in [',', ';']:
        search_char_count = word.count(search_char)
        punctuation_count[search_char] += search_char_count
    search_for_single_quotes(word)
    search_for_hyphens(word)

Out:
AttributeError: 'list' object has no attribute 'find'

Answer 1

Given the following input:

df = pd.DataFrame(['I love, pizza, hamberget and chips!!.', 'I like drink beer,, cofee and water!.'], columns=['content'])

                                content
0   I love, pizza, hamberget and chips!!.
1   I like drink beer,, cofee and water!.

Try this code:

count = lambda l1,l2: sum([1 for x in l1 if x in l2])

df['count_punct'] = df.content.apply(lambda s: count(s, string.punctuation))

and gives:

                                 content  count_punct
0  I love, pizza, hamberget and chips!!.            5
1  I like drink beer,, cofee and water!.            4

If you want to accumulate the punctuation marks of each line in a list:

accumulate = lambda l1,l2: [x for x in l1 if x in l2]

df['acc_punct_list'] = df.content.apply(lambda s: accumulate(s, string.punctuation))

and gives:

                                 content  count_punct   acc_punct_list
0  I love, pizza, hamberget and chips!!.            5  [,, ,, !, !, .]
1  I like drink beer,, cofee and water!.            4     [,, ,, !, .]

If you want to accumulate the punctuation marks of each line in a dict and transpose each element into dataframe columns:

df['acc_punct_dict'] = df.content.apply(lambda s: {k:v for k, v in Counter(s).items() if k in string.punctuation})

                                 content            acc_punct_dict
0  I love, pizza, hamberget and chips!!.  {',': 2, '!': 2, '.': 1}
1  I like drink beer,, cofee and water!.  {',': 2, '!': 1, '.': 1}

now expanding the dictionary in the columns of the df:

df_punct = df.acc_punct_dict.apply(pd.Series)

   ,  !  .
0  2  2  1
1  2  1  1

If you want to combine the new dataframe with the starting one, you just need to do:

df_res = pd.concat([df, df_punct], axis=1)

and gives:

                                 content            acc_punct_dict  ,  !  .
0  I love, pizza, hamberget and chips!!.  {',': 2, '!': 2, '.': 1}  2  2  1
1  I like drink beer,, cofee and water!.  {',': 2, '!': 1, '.': 1}  2  1  1

Note: If you don't care about the column with the dictionary you can delete it by df_res.drop('acc_punct_dict', axis=1)