Search code examples
pythonpandasdataframesparse-matrix

Sparse DataFrame returns AttributeError

I have created the following DataFrame:

trains = np.arange(100)
tresholds = [10, 20, 30, 40, 50, 60]
tuples = []
for i in trains:
    for j in tresholds:
        tuples.append((i, j))

index = pd.MultiIndex.from_tuples(tuples, names=['trains', 'tresholds'])
matrix = np.empty((len(index), len(trains)))
matrix.fill(np.nan)
df = pd.DataFrame(matrix, index=index, columns=trains, dtype=float)

This DataFrame is filled using df.loc[(x, y), z] indexing, but it contains more NaN than actual numbers, so I wanted to create a Sparse DataFrame. But df.to_sparse() gives me this error (full trace).

Solution

  • All nan columns are buggy ATM in this kind of conversion. If you already had a SparseFrame adding a nan column would work however.

    If you did this:

    df.iloc[0] = 0
df.to_sparse()

    works.