Pandas - store numpy array in a dataframe column which is a result of function

Question

I have a pandas dataframe with a column allTexts which stores a bunch of text information for each row. I am trying to apply a custom function which returns 3 values given the input text. I then want to store these 3 output values in a new dataframe column - ideally as a numpy array for each row. I do it with apply(), the code completes successfully but it doesn't actually change values.

#stub for creating a dataframe
df = pd.DataFrame({'allText':['Hateful text. This is bad', 'Text about great stuff', ' ']})

#set a placeholder - just 3 zeros for each record
df['Sentiments'] = df['allText'].apply(lambda x: np.zeros(3))

#function definition. It is a textblob library function, which gives me back sentiment scores for each text
def getTextSentiments(text):
    blob = TextBlob(text)
    pos = 0
    neg = 0
    neutral = 0
    count = 0
    for sentence in blob.sentences:
        sentiment = sentence.sentiment.polarity
        if sentiment > 0.1:
            pos +=1
        elif sentiment > -0.1:
            neutral +=1
        else:
            neg +=1
        count+=1
    if count == 0:
        count = 1
    return numpy.array([pos/count, neutral/count, neg/count])

#apply function only for non-empty texts and override 3 zeros in sentiments column with real 3 values
df[df["allText"]!=" "]['Sentiments'] = df[df["allText"]!=" "]["allText"].apply(getTextSentiments)

After this code completes without any error I still end up with same value of all zeros in my Sentiments column.

MVP to demonstrate it doesn't work even with single record:

df[df["allText"]!=" "].iloc[0]['Sentiments']
array([ 0.,  0.,  0.])
test = getTextSentiments(df[df["allText"]!=" "].iloc[0]['allText'])

test
Out[64]: (0.4166666666666667, 0.5, 0.08333333333333333)
df[df["allText"]!=" "].iloc[0]['Sentiments'] = test

df[df["allText"]!=" "].iloc[0]['Sentiments']
Out[75]: array([ 0.,  0.,  0.])

Any advice on what am I doing wrong?

Answer 1

Can you try the following?

df.Sentiments = df.apply(lambda x: x.Sentiments if x.allText ==' ' else getTextSentiments(x.allText), axis=1)

Using a dummy getTextSentiments function for test:

df = pd.DataFrame({'allText':['Hateful text. This is bad', 'Text about great stuff', ' ']})

#set a placeholder - just 3 zeros for each record
df['Sentiments'] = df['allText'].apply(lambda x: np.zeros(3))
def getTextSentiments(text):
    return (0.4166666666666667, 0.5, 0.08333333333333333)
df.Sentiments = df.apply(lambda x: x.Sentiments if x.allText ==' ' else getTextSentiments(x.allText), axis=1)
df
Out[181]: 
                     allText                                      Sentiments
Out[181]: 
                     allText                                      Sentiments
0  Hateful text. This is bad  (0.4166666666666667, 0.5, 0.08333333333333333)
1     Text about great stuff  (0.4166666666666667, 0.5, 0.08333333333333333)
2                                                            [0.0, 0.0, 0.0]