Search code examples
pythonpandasnumpyvectordot-product

Why doesn't dot product x@A work if x is a pandas Series?

  • A is a 5x5 square matrix pandas DataFrame
  • x is a 5 (one-dimensional) vector pandas Series

x@A returns error ValueError: matrices are not aligned even though they clearly both meet the requirement for dot product multiplication, having the same outer-dimension, 5.

whereas x.values @ A works, returning the expected scalar, simply because x has been changed from a pandas Series to a numpy array

Why is the dot symbol @ so picky with pandas?

Solution

  • See the documentation:

    In addition, the column names of DataFrame and the index of other must contain the same values, as they will be aligned prior to the multiplication.

    So the error is not about dimensions but rather about non-matching indices. See the following example:

    import pandas as pd

df = pd.DataFrame([[1,2],[3,4]], columns=list('ab'))
s = pd.Series([5,6])

# df @ s                                      # --> doesn't work
print(df.values @ s)                          # --> works because no column names involved
print(df.rename({'a':0, 'b':1}, axis=1) @ s)  # --> works because indices match

    or the other way round

    df = pd.DataFrame([[1,2],[3,4]], index=list('ab'))
s = pd.Series([5,6])

# s @ df                             # --> doesn't work
print(s @ df.values)                 # --> works because no column names involved
print(s @ df.reset_index(drop=True)) # --> works because indices match