Search code examples
pythonpandasdata-preprocessing

Python Pandas: Drop rows from data frame if list of string value == [none]

I have a column in my data frame that contains lists of values.

 Tags
 [marvel, comics, comic, books, nerdy]
 [new, snapchat, version, snap, inc]
 [none]
 [new, york, times, ny, times, nyt, times]
 [today, show, today, show, today]
 [none]
 [mark, wahlberg, marky, mark]

I don't know how to remove this [none] list from the data frame. I tried,

 us_videos = us_videos.drop(us_videos.index[us_videos.tags == 'none'])

But this only working when I turn the column into string. How to achieve this?

Solution

  • New Answer

    OP wanted to remove 'none' from sub-lists and remove rows with only 'none'

    us_videos.tags.explode().pipe(lambda s: s[s != 'none']).groupby(level=0).agg(list)

0        [marvel, comics, comic, books, nerdy]
1          [new, snapchat, version, snap, inc]
3    [new, york, times, ny, times, nyt, times]
4            [today, show, today, show, today]
6                [mark, wahlberg, marky, mark]
Name: tags, dtype: object

    A more pythonic way

    dat = {}
for k, v in us_videos.tags.iteritems():
    for x in v:
        if x != 'none':
            dat.setdefault(k, []).append(x)

pd.Series(dat, name='tags')

0        [marvel, comics, comic, books, nerdy]
1          [new, snapchat, version, snap, inc]
3    [new, york, times, ny, times, nyt, times]
4            [today, show, today, show, today]
6                [mark, wahlberg, marky, mark]
Name: tags, dtype: object

    With assignment expressions in a comprehension

    pd.Series({
    k: X for k, v in us_videos.tags.iteritems()
    if (X:=[*filter('none'.__ne__, v)])
}, name='tags')

0        [marvel, comics, comic, books, nerdy]
1          [new, snapchat, version, snap, inc]
3    [new, york, times, ny, times, nyt, times]
4            [today, show, today, show, today]
6                [mark, wahlberg, marky, mark]
Name: tags, dtype: object

    OLD ANSWERS

    explode

    us_videos[us_videos.tags.explode().ne('none').any(level=0)]

                                        tags
0      [marvel, comics, comic, books, nerdy]
1        [new, snapchat, version, snap, inc]
3  [new, york, times, ny, times, nyt, times]
4          [today, show, today, show, today]
6              [mark, wahlberg, marky, mark]

    list.__ne__

    us_videos[us_videos.tags.map(['none'].__ne__)]

                                        tags
0      [marvel, comics, comic, books, nerdy]
1        [new, snapchat, version, snap, inc]
3  [new, york, times, ny, times, nyt, times]
4          [today, show, today, show, today]
6              [mark, wahlberg, marky, mark]