Not able to split data frame in numpy

Question

Not able to use numpy split function to allocate subsets of dataframe to

cols =["fLength","fWidth","fSize","fConc","fConcl","fAsym","fM3Long","fAlpha","fDist","class"]
df = pd.read_csv("magic04.data",names = cols)
df['class'] = (df['class']=='g').astype(int)

train, valid, test = np.split(df.sample(frac=1), [int(0.6*len(df)) , int(0.8*len(df)), ])

KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3628             try:
-> 3629                 return self._engine.get_loc(casted_key)
   3630             except KeyError as err:

17 frames
KeyError: 0

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3629                 return self._engine.get_loc(casted_key)
   3630             except KeyError as err:
-> 3631                 raise KeyError(key) from err
   3632             except TypeError:
   3633                 # If we have a listlike key, _check_indexing_error will raise

Tried reading the documentation but didnt find anything useful.

Answer 1

The error in your code is that you are trying to use a numpy routine with a pandas data frame. The best way to approach this is to convert your df.sample into a numpy array and then use np.split().

Try this - it runs perfectly well on my VSCode:

npsample=np.array(df.sample(frac=1))
train, valid, test = np.split(npsample, [int(0.6*len(npdata)) , int(0.8*len(npdata)), ])