Search code examples
pythonpandasdataframeassignbroadcasting

How to broadcast and assign a series of values across all columns in a Pandas dataframe?


I know this must be easy but I can't figure it out or find an existing answer on this...

Say I have this dataframe...

>>> import pandas as pd
>>> import numpy as np
>>> dates = pd.date_range('20130101', periods=6)
>>> df = pd.DataFrame(np.nan, index=dates, columns=list('ABCD'))
>>> df
             A   B   C   D
2013-01-01 NaN NaN NaN NaN
2013-01-02 NaN NaN NaN NaN
2013-01-03 NaN NaN NaN NaN
2013-01-04 NaN NaN NaN NaN
2013-01-05 NaN NaN NaN NaN
2013-01-06 NaN NaN NaN NaN

It's easy to set the values of one series...

>>> df.loc[:, 'A'] = pd.Series([1,2,3,4,5,6], index=dates)
>>> df
            A   B   C   D
2013-01-01  1 NaN NaN NaN
2013-01-02  2 NaN NaN NaN
2013-01-03  3 NaN NaN NaN
2013-01-04  4 NaN NaN NaN
2013-01-05  5 NaN NaN NaN
2013-01-06  6 NaN NaN NaN

But how do I set the values of all columns using broadcasting?

>>> default_values = pd.Series([1,2,3,4,5,6], index=dates)
>>> df.loc[:, :] = default_values
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/billtubbs/anaconda/envs/py36/lib/python3.6/site-packages/pandas/core/indexing.py", line 189, in __setitem__
    self._setitem_with_indexer(indexer, value)
  File "/Users/billtubbs/anaconda/envs/py36/lib/python3.6/site-packages/pandas/core/indexing.py", line 651, in _setitem_with_indexer
    value=value)
  File "/Users/billtubbs/anaconda/envs/py36/lib/python3.6/site-packages/pandas/core/internals.py", line 3693, in setitem
    return self.apply('setitem', **kwargs)
  File "/Users/billtubbs/anaconda/envs/py36/lib/python3.6/site-packages/pandas/core/internals.py", line 3581, in apply
    applied = getattr(b, f)(**kwargs)
  File "/Users/billtubbs/anaconda/envs/py36/lib/python3.6/site-packages/pandas/core/internals.py", line 940, in setitem
    values[indexer] = value
ValueError: could not broadcast input array from shape (6) into shape (6,4)

Other than these ways:

>>> for s in df:
...     df.loc[:, s] = default_values
... 

Or:

>>> df.loc[:, :] = np.vstack([default_values]*4).T

UPDATE:

Or:

>>> df.loc[:, :] = default_values.values.reshape(6,1)

Solution

  • Use numpy broadcasting

    s =  pd.Series([1,2,3,4,5,6], index=dates)
    df.loc[:,:] = s.values[:,None]
    

    Using index matching

    df.loc[:] = pd.concat([s]*df.columns.size, axis=1)