Search code examples
c++arraysdynamic-memory-allocationhdf5

Dynamic memory allocation for 4 Dimensional c++ array to create HDF5 dataset


I have a program in QT which creates a HDF5 dataset and saves it in disk.

At first I defined this array:

int data[dataNumber][dataFilter][dataWidth][dataHeight];

where dataNumber=400000,dataFilter=1,dataWidth=2,dataHeight=2 and in this loop I set a identical values in this array (of course for simplicity):

for (int i = 0; i < dataNumber; i++)
    for (int j = 0; j < dataFilter; j++)
        for (int k = 0; k < dataWidth; k++)
            for (int l = 0; l < dataHeight; l++)
                data[i][j][k][l]=2;

after that I used these commands to create HDF5 dataset:

// Create a new file using the default property lists.
H5File file(FILE_NAME, H5F_ACC_TRUNC);

datadims[0] = dataNumber;
datadims[1] = dataFilter;
datadims[2] = dataWidth;
datadims[3] = dataHeight;

IntType datatype( PredType::NATIVE_INT );
datatype.setOrder( H5T_ORDER_LE );
DataSpace dataspace(4, datadims);

// Create the dataset.
DataSet dataset = file.createDataSet("data", datatype, dataspace);
dataset.write(data, PredType::NATIVE_INT);

and every thing was ok.

But when I set dataWidth=4,dataHeight=4 ,segmentation fault occured and I think the reserved size of the stack causes this problem and I should use Dynamic memory allocation.

So I changed the data array as follows to overcome with that:

int ****data ;
data = new int***[dataNumber];
for( int i = 0 ; i < dataNumber ; i++ )
{
    data[i] = new int**[dataFilter] ;
    for( int j = 0 ; j < dataFilter ; j++ )
    {
        data[i][j] = new int*[dataWidth] ;
        for( int k = 0 ; k < dataWidth ; k++ )
            data[i][j][k] = new int[ dataHeight ] ;
     }
}

But after using this:

DataSet dataset = file.createDataSet("data", datatype, dataspace);
dataset.write(data, PredType::NATIVE_INT);

the HDF5 dataset had a lot of wrong numbers like -1865382688 and -1720619824 instead of true number=2.

Now I don't know how can I solve this problem. I hope someone can help.


Solution

  • I would suggest to allocate one contiguous chunk of memory, like this:

    int *data = new int[dataNumber * dataFilter * dataWidth * dataHeight];
    

    This means that you will need to call the new operator once, instead of calling it thousands of times. This will also make the cleanup of all this memory easy with just making one call:

    delete []data;
    

    instead of calling delete thousands of times again. After that, you can index into your array with something like this:

    data[((i*dataFilter + j)*dataWidth + k)*dataHeight + l];