Search code examples
pythonhdf5h5py

Python HDF5 expand array


I'm trying to expand the dimensions of <HDF5 dataset "time_data": shape (2048000, 63), type "<f4"> to shape (2048000, 64) . In the last column I'd like to add then the information from "<HDF5 dataset "time_data": shape (2048000, 1), type "<f4">". But I'm quite clueless how to do that.

So far I've tried:

mic['time_data'].resize(64, axis = 0)

But it returns me a <HDF5 dataset "time_data": shape (64, 63), type "<f4">

I'm very new to HDF5 files, so please tell me if you need more information than I've given to you!


Solution

  • After re-reading your post, I realized your dataset was resizable (otherwise h5py would have thrown an error on the resize). You can confirm by comparing the .shape attribute to the .maxshape attribute. Note: None means no size limit on that axis.

    [For future reference, if you want a resizable dataset, include the maxshape=() parameter when you create it. For this particular dataset, use maxshape=(2048000, None) if you only want to resize axis 1.]

    Your unexpected result was caused by a small error in your resize declaration. To go from shape (2048000, 63) to shape (2048000, 64) it should be: mic['time_data'].resize(64, axis=1) or mic['time_data'].resize((2048000, 64))

    Updated answer
    Here is an expanded answer that shows how to add the the data from 'assembledH5Trigger.h5' to 'assembledH5TEST.h5'. For reference, it prints the .shape and .maxshape parameters before it copies the data from trigger to mic files.

    with h5py.File('assembledH5TEST.h5', 'r+') as mic, \
         h5py.File('assembledH5Trigger.h5', 'r') as trigger:
    
        print(f"mic['time_data'] shape={mic['time_data'].shape}; " + \
              f" maxshape={mic['time_data'].maxshape}")          
        print(f"trigger['time_data'] shape={trigger['time_data'].shape}; " + \
              f"maxshape={trigger['time_data'].maxshape}")          
        mic['time_data'].resize((2048000, 64))
        mic['time_data'][:,63] = trigger['time_data'][:,0]