Search code examples
pythonarraysnumpydynamicindexoutofboundsexception

Are there dynamic arrays in numpy?


Let's say I create 2 numpy arrays, one of which is an empty array and one which is of size 1000x1000 made up of zeros:

import numpy as np;
A1 = np.array([])
A2 = np.zeros([1000,1000])

When I want to change a value in A2, this seems to work fine:

A2[n,m] = 17

The above code would change the value of position [n][m] in A2 to 17.

When I try the above with A1 I get this error:

A1[n,m] = 17

IndexError: index n is out of bounds for axis 0 with size 0

I know why this happens, because there is no defined position [n,m] in A1 and that makes sense, but my question is as follows:

Is there a way to define a dynamic array without that updates the array with new rows and columns if A[n,m] = somevalue is entered when n or m or both are greater than the bound of an Array A?

It doesn't have to be in numpy, any library or method that can update array size would be awesome. If it is a method, I can imagine there being an if loop that checks if [n][m] is out of bounds and does something about it.

I am coming from a MATLAB background where it's easy to do this. I tried to find something about this in the documentation in numpy.array but I've been unsuccessful.

EDIT: I want to know if some way to create a dynamic list is possible at all in Python, not just in the numpy library. It appears from this question that it doesn't work with numpy Creating a dynamic array using numpy in python.


Solution

  • This can't be done in numpy, and it technically can't be done in MATLAB either. What MATLAB is doing behind-the-scenes is creating an entire new matrix, then copying all the data to the new matrix, then deleting the old matrix. It is not dynamically resizing, that isn't actually possible because of how arrays/matrices work. This is extremely slow, especially for large arrays, which is why MATLAB nowadays warns you not to do it.

    Numpy, like MATLAB, cannot resize arrays (actually, unlike MATLAB it technically can, but only if you are lucky so I would advise against trying). But in order to avoid the sort of confusion and slow code this causes in MATLAB, numpy requires that you explicitly make the new array (using np.zeros) then copy the data over.

    Python, unlike MATLAB, actually does have a truly resizable data structure: the list. Lists still require there to be enough elements, since this avoids silent indexing errors that are hard to catch in MATLAB, but you can resize an array with very good performance. You can make an effectively n-dimensional list by using nested lists of lists. Then, once the list is done, you can convert it to a numpy array.