Search code examples
pythonnumpyscipyadditionsparse-matrix

How to efficiently add sparse matrices in Python

I want to know how to efficiently add sparse matrices in Python.

I have a program that breaks a big task into subtasks and distributes them across several CPUs. Each subtask yields a result (a scipy sparse matrix formatted as: lil_matrix).

The sparse matrix dimensions are: 100000x500000 , which is quite huge, so I really need the most efficient way to sum all the resulting sparse matrices into a single sparse matrix, using some C-compiled method or something.

Solution

  • Have you tried timing the simplest method?

    matrix_result = matrix_a + matrix_b

    The documentation warns this may be slow for LIL matrices, suggesting the following may be faster:

    matrix_result = (matrix_a.tocsr() + matrix_b.tocsr()).tolil()