Search code examples
hdf5hdfhdfql

HDFql Adding Data to Existing Dataset


I want to create a HDF5 file in C++ and dynamically add data to a dataset contained in that file. I have been having some trouble getting this work however, with this minimal example as my current best attempt (based mainly on an answer by SOG):

#include <stdio.h>
#include <HDFql.hpp>

int main (int argc, const char * argv[]) {
    char script[1024];

    //Create HDF5 file
    HDFql::execute("CREATE TRUNCATE FILE /tmp/test.h5");
    HDFql::execute("USE FILE /tmp/test.h5");

    //Generate some arbitrary data
    int data_size = 10;
    int data[data_size];
    for (int i=0; i<data_size; i++) {data[i]=i+data_size;}

    //Create a dataset "dset"
    HDFql::execute("CREATE CHUNKED DATASET dset AS INT(UNLIMITED)");
    //In the loop, add one element to dset on each iteration
    for(int i=0; i<10; i++)
    {
        HDFql::execute("ALTER DIMENSION dset TO (+1)");
        sprintf(script, "INSERT INTO dset(-1:1:1:1) VALUES(%d)", i);
        HDFql::execute(script);
    }
    //Add an array of values to dset
    sprintf(script, "ALTER DIMENSION dset TO (%d)", data_size);
    HDFql::execute(script);
    sprintf(script, "INSERT INTO dset(-1:1:1:1) VALUES(%d)", data_size);
    HDFql::execute(script);

    //Read all values in dset and print to console
    HDFql::execute("SELECT FROM dset");
    int cnt=0;
    while(HDFql::cursorNext(NULL) == HDFQL_SUCCESS)
    {
        printf("Value at %d: %d\n", cnt, *HDFql::cursorGetInt(NULL));
        cnt++;
    }

    HDFql::execute("CLOSE FILE");
}

I would expect the output of the console to be:

Value at 0: 0
Value at 1: 1
Value at 2: 2
...
Value at 19: 19

Instead the output is:

Value at 0: 10

And indeed, inspecting the HDF5 file shows that it contains only one value, 10. So what is wrong with my code and why? Also, a short explanation of the hyperslab (which I am using quite blindly here) or a link to documentation explaining it would be very appreciated.

Many thanks!

Edit

Here is a working code example based on the answer by @SOG:

#include <stdio.h>
#include <HDFql.hpp>

int main (int argc, const char * argv[]) {
    char script[1024];

    HDFql::execute("CREATE TRUNCATE FILE /tmp/test.h5");
    HDFql::execute("USE FILE /tmp/test.h5");

    int data_size = 10;
    int data[data_size];
    for (int i=0; i<data_size; i++) {data[i]=i+data_size;}

    HDFql::execute("CREATE CHUNKED DATASET dset AS INT(UNLIMITED)");
    for(int i=0; i<10; i++)
    {
        if(i>0) {HDFql::execute("ALTER DIMENSION dset TO +1");}
        sprintf(script, "INSERT INTO dset(-1:1:1:1) VALUES(%d)", i);
        HDFql::execute(script);
    }
    sprintf(script, "ALTER DIMENSION dset TO +%d", data_size);
    HDFql::execute(script);
    sprintf(script, "INSERT INTO dset(-%d:1:1:%d) VALUES FROM MEMORY %d",
            data_size, data_size, HDFql::variableTransientRegister(data));
    HDFql::execute(script);


    int cnt=0;
    HDFql::execute("SELECT FROM dset");
    while(HDFql::cursorNext(NULL) == HDFQL_SUCCESS)
    {
        printf("Value at %d: %d\n", cnt, *HDFql::cursorGetInt(NULL));
        cnt++;
    }

    HDFql::execute("CLOSE FILE");
}

Solution

  • According to HDFql release notes, operation ALTER DIMENSION has changed syntax in version 2.1.0 (i.e. the operation does not require parenthesis to surround dimensions anymore).

    Therefore, change your code to:

    HDFql::execute("ALTER DIMENSION dset TO +1");
    

    Additional information about this operation as well as hyperslabs can be found in HDFql reference manual.