I have a dataframe where one of the columns have some rows with an array value instead of a single int64 value. I want to drop all such rows.
I am using the following code to do this but this is not working (for obvious reasons as it is being compared to a string).
handover_data.drop(handover_data[handover_data['S-PCI'] == '[105 106]'].index, inplace=True)
In the dataframe it should either have 105 or 106 not both but some of the palces have [105 106]
What are the ways to compare this to check if there is an array instead of the expected value:
Data set looks like the following:
S-Cell ID N-Cell ID S-PLMN S-PCI N-PCI S-BW N-BW \
73 257 0 105 105 106 2147483647 2147483647
S-EARFCN N-EARFCN
73 3025 3025 30102
Elapsed RT Time (ms) RSRP-105 RSRP-106 RSRQ-105 RSRQ-106
73 41846000000 2947094.0 -84 -90 -4 -14
EDIT:
s_Cell = 105
for i, j in hd_data.iterrows():
if(hd_data.at[i,'S-PCI'].all() != s_Cell):
hd_data.at[i,'H_Event'] = 1
This is failing with following error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-399-d4c47a34a73e> in <module>
19 print(i)
20 print(handover_data.at[i,'S-PCI'])
---> 21 if(handover_data.at[i,'S-PCI'].all() != starting_Cell):
22 handover_data.at[i,'Handover_Event'] = 1
23 #handover_data.at[i,'Time_to_Handover'] = handover_data.at[i,'TimeInterval']-last_HO_time
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
you can filter by the values you want like
wanted_values = [105, 106]
handover_data = handover_data[handover_data['S-PCI'].isin(wanted_values)]
if you want to remove the items that are specifically a list then it'd be a bit more resource intensive
import numpy as np
handover_data = handover_data.apply(lambda x: np.nan if isinstance(x['S-PCI'], list) else x).dropna(subset=['S-PCI'])