I have a code of matrix multiplictaion using numpy memmap
def test_memmap():
fA = np.lib.format.open_memmap('A.npy', dtype='uint8', mode='r+')
fB = np.lib.format.open_memmap('B.npy', dtype='uint8', mode='r+')
#need to predefine size of result?
#fC = np.memmap('C.npy', dtype='uint16', mode='w+', shape=(rows,rows))
# print fA
# print fB
#where it stored if not gived filename?
t0= time.time()
fC= np.dot(fA,fB)
print (time.time()-t0)
print fC.filename
print type(fC)
# print fC[12:10]
# print fC
where memmap file fC is located after operation? And how to specify save path?
How can I point where to save fC(not to store it in memory)? can type and size of array be auto detected?
Although fC
is an instance of numpy.core.memmap.memmap,
it isn't associated with any file:
print(type(fC))
# <class 'numpy.core.memmap.memmap'>
print(fC._mmap)
# None
print(fC.filename)
# None
The reason why fC
is an instance of memmap is because np.dot
, like most NumPy functions, tries to return an array of the same type as its arguments. So the memmap.__array_wrap__
method gets called. The __array_finalize__
method which gets called last, sets the _mmap
, filename
, offset
and mode
attributes to None.
You can see __array_finalize__
getting called if you step through the code with a debugger like pdb.
So although fC
is a memmap instance, it exists in memory only.
To compute np.dot
with the result written to a memmap:
fC = np.memmap('/tmp/C.npy', dtype='float', mode='w+', shape=(Arows, Bcols))
np.dot(fA,fB, out=fC)
print fC.filename
# /tmp/C.npy
print type(fC)
# <class 'numpy.core.memmap.memmap'>