How to delete rows with less than a certain amount of items or strings with Pandas?

I have searched a lot but couldn't find a solution to this particular case. I want to remove any rows that contains less than 3 strings or items in the lists. My issues will be addressed more clearly further down.

I'm preparing a LDA topic modelling with a large Swedish database in pandas and have limited the test case to 1000 rows. I'm only concerned with a specific column and my approach so far has been as follows:

con = sqlite3.connect('/Users/mo/EXP/NAV/afm.db')
sql = """
select * from stillinger limit 1000
"""
dfs = pd.read_sql(sql, con)

plb = """
select PLATSBESKRIVNING from stillinger limit 1000
"""
dfp = pd.read_sql(plb, con);dfp

Then I've defined a regular expression where the first argument removes any meta characters while keeping the Swedish and Norwegian language specific letters. The second argument removes words < 3:

rep = {
   'PLATSBESKRIVNING': {
     r'[^A-Za-zÅåÄäÖöÆØÅæøå]+': ' ',
     r'\W*\b\w{1,3}\b': ' '}
}

p0 = (pd.DataFrame(dfp['PLATSBESKRIVNING'].str.lower()).replace(rep, regex=True).
  drop_duplicates('PLATSBESKRIVNING').reset_index(drop=True));p0

PLATSBESKRIVNING
0   medrek rekrytering söker uppdrag manpower h...
1   familj barn tjejer kille söker pair ...
2   uppgift blir tillsammans medarbetare leda ...
3   behov operasjonssykepleiere langtidsoppdr...
4   detta perfekta jobbet arbetstiderna vardaga...
5   familj paris barn söker älskar barn v...
6   alla inom cafe restaurang förekommande arbets...
.
.

Creating a pandas Series:

s0 = p0['PLATSBESKRIVNING']

Then:

ts = s0.str.lower().str.split();ts

0      [medrek, rekrytering, söker, uppdrag, manpower...
1      [familj, barn, tjejer, kille, söker, pair, vil...
2      [uppgift, blir, tillsammans, medarbetare, leda...
3      [behov, operasjonssykepleiere, langtidsoppdrag...
4      [detta, perfekta, jobbet, arbetstiderna, varda...
5      [familj, paris, barn, söker, älskar, barn, vil...
6      [alla, inom, cafe, restaurang, förekommande, a...
7      [diskare, till, cafe, dubbel, sökes, arbetet, ...
8      [diskare, till, thelins, konditori, sökes, arb...

Removing the stop words from the database:

r = s0.str.split().apply(lambda x: [item for item in x if item not in mswl]);r

0      [uppdrag, bemanningsföretag, erbjuds, tillägg,...
1      [föräldrarna, citycentre, stort, tomt, mamman,...
2      [utveckla, övergripande, strategiska, frågor, ...
3                   [erfaring, sykepleier, legitimasjon]
4      [arbetstiderna, vardagar, härliga, människor, ...
5      [paris, utav, badrum, båda, yngsta, endast, fö...
6                       [förekommande, emot, utbildning]
7                                                     []
8                                              [thelins]
9      [paris, baby, månader, våning, delar, badrum, ...

Creating a new DataFrame and removing the empty brackets:

dr = pd.DataFrame(r)
dr0 = dr[dr.astype(str)['PLATSBESKRIVNING'] != '[]'].reset_index(drop=True); dr0

PLATSBESKRIVNING
0   [uppdrag, bemanningsföretag, erbjuds, tillägg,...
1   [föräldrarna, citycentre, stort, tomt, mamman,...
2   [utveckla, övergripande, strategiska, frågor, ...
3   [erfaring, sykepleier, legitimasjon]
4   [arbetstiderna, vardagar, härliga, människor, ...
5   [paris, utav, badrum, båda, yngsta, endast, fö...
6   [förekommande, emot, utbildning]
7   [thelins]
8   [paris, baby, månader, våning, delar, badrum, ...

Maintaining the string:

dr1 = dr0['PLATSBESKRIVNING'].apply(str);  len(dr1),type(dr1), dr1

0      ['uppdrag', 'bemanningsföretag', 'erbjuds', 't...
1      ['föräldrarna', 'citycentre', 'stort', 'tomt',...
2      ['utveckla', 'övergripande', 'strategiska', 'f...
3             ['erfaring', 'sykepleier', 'legitimasjon']
4      ['arbetstiderna', 'vardagar', 'härliga', 'männ...
5      ['paris', 'utav', 'badrum', 'båda', 'yngsta', ...
6                 ['förekommande', 'emot', 'utbildning']
7                                            ['thelins']
8      ['paris', 'baby', 'månader', 'våning', 'delar'...

My issue now is that I want to remove any rows that contains less than 3 strings in the lists, e.g row 3, 6 and 7. Desired result would be like this:

0      ['uppdrag', 'bemanningsföretag', 'erbjuds', 't...
1      ['föräldrarna', 'citycentre', 'stort', 'tomt',...
2      ['utveckla', 'övergripande', 'strategiska', 'f...
3      ['arbetstiderna', 'vardagar', 'härliga', 'männ...
4      ['paris', 'utav', 'badrum', 'båda', 'yngsta', ...
5      ['paris', 'baby', 'månader', 'våning', 'delar'...
.
.

How can I obtain this? I'm also wondering if this could be done in a neater way? My approach seems so clumsy and cumbersome.

I would also like to remove both indexes and column name for the LDA topic modelling such that I could write it to a text file without the header and the digits of indexes. I have tried:

dr1.to_csv('LDA1.txt',header=None,index=False)

But this wraps quotation marks "['word1', 'word2', 't.. ]" to the each list of strings in the file.

Any suggestions would be much appreciated.

Best regards Mo

Solution

Just measure the number of items in the list and filter the rows with length lower than 3

dr0['length'] = dr0['PLATSBESKRIVNING'].apply(lambda x: len(x))
cond = dr0['length'] > 3
dr0 = dr0[cond]