Support for basic datatypes in H5Attributes?

Question

I am trying out the beta hdf5 toolkit of ilnumerics.

Currently I see H5Attributes support only ilnumerics arrays. Is there any plan to extend it for basic datatypes (such as string) as part of the final release?

Does ilnumerics H5 wrappers provide provision for extending any functionality to a particular datatype?

Answer 1

ILNumerics internally uses the official HDF5 libraries from the HDF Group, of course. H5Attributes in HDF5 correspond to datasets with the limitation of being not capable of partial I/O. Besides that, H5Attributes are plain arrays! Support for basic (scalar) element types is given by assuming the array stored to be scalar.

Strings are a complete different story: strings in general are variable length datatypes. In terms of HDF5 strings are arrays of element type Char. The number of characters in the string determines the length of the array. In order to store a string into a dataset or attribute, you will have to store its individual characters as elements of the array. In ILNumerics, you can convert your string into ILArrray or ILArray (for ASCII data) and store that into the dataset/ attribute.

Please consult the following test case which stores a string as value into an attribute and reads the content back into a string.

Disclaimer: This is part of our internal test suite. You will not be able to compile the example directly, since it depends on the existence of several functions which may are not available. However, you will be able to understand how to store strings into datasets and attributes:

public void StringASCIAttribute() {
    string file = "deleteA0001.h5";
    string val = "This is a long string to be stored into an attribute.\r\n";
    // transfer string into ILArray<Char>
    ILArray<Char> A = ILMath.array<Char>(' ', 1, val.Length);
    for (int i = 0; i < val.Length; i++) {
        A.SetValue(val[i], 0, i);
    }
    // store the string as attribute of a group
    using (var f = new H5File(file)) {
        f.Add(new H5Group("grp1") {
            Attributes = { 
                { "title", A }
            }
        });
    }
    // check by reading back
    // read back 
    using (var f = new H5File(file)) {
        // must exist in the file
        Assert.IsTrue(f.Get<H5Group>("grp1").Attributes.ContainsKey("title"));
        // check size 
        var attr = f.Get<H5Group>("grp1").Attributes["title"];
        Assert.IsTrue(attr.Size == ILMath.size(1, val.Length));
        // read back 
        ILArray<Char> titleChar = attr.Get<Char>();
        ILArray<byte> titleByte = attr.Get<byte>();

        // compare byte values (sum) 
        int origsum = 0;
        foreach (var c in val) origsum += (Byte)c;
        Assert.IsTrue(ILMath.sumall(ILMath.toint32(titleByte)) == origsum);

        StringBuilder title = new StringBuilder(attr.Size[1]);
        for (int i = 0; i < titleChar.Length; i++) {
            title.Append(titleChar.GetValue(i));
        }
        Assert.IsTrue(title.ToString() == val);
    }
}

This stores arbitrary strings as 'Char-array' into HDF5 attributes and would work just the same for H5Dataset.