Search code examples
c++boosthdf5unsigned-integer

HDF5 write out 32 bit unsigned int to 64 bits on disk and read into 32 bit unsigned int


I am trying to write out a vector of 32-bit unsigned int to 64-bit unsigned long on disk using HDF5; and then read that back into 32 bit unsigned int in memory. For this my read-write functions look as follows (I have all the well-defined functions for push_back, resize, etc. implemented for my 32-bit unsigned int container, my_container):

bool write(const my_container<unsigned int>& p, const std::string& datapath, const H5::DateType& datatype):
    try {
       const hsize_t h5size[1] = { p.size() };
       const H5::DataSpace h5space(1, h5size);
       const H5::DataSet h5set = fileptr_->createDataSet(datapath, datatype, h5space);
       //-Tried using void* here for data, no difference
       const unsigned int* data = &(*p.begin()); 
       h5set.write(data, datatype);
    } 
    catch( H5::Exception& ) {
       //-Handle exception here
       return false;
    }
    return true;
}

read(my_container<unsigned int>& p, const H5::DataType& datatype, const std::string& datapath) {
    H5::DataType h5set_datatype = h5set.getDataType();
    const std::size_t size = (std::size_t)h5space.getSimpleExtentNpoints();
    try {
       if(h5set_datatype == H5::PredType::NATIVE_UINT64 && datatype == H5::PredType::NATIVE_UINT32 ) {
           typedef unsigned long long u64; 
           typedef std::vector<u64> u64vec;
           u64vec ivector;
           ivector.resize(size);
           void* data = (void*)(&(*ivector.begin()));
           h5set.read(data, h5set_datatype);
           p.resize(0);
           BOOST_FOREACH(const u64 &v, ivector) {
              //-I've handled the cast below using numeric cast separately
              p.push_back(v); 
           }
       } //-End compare datatypes on disk and memory
    } //-End try
    catch(const H5::Exception &e) {
        //-Handle exception
        return false;
    }
    return true;
}

I call write with arguments: const-reference to my_container, H5::Pred::NATIVE_UINT64, and read with arguments: reference to my_container andH5::Pred::NATIVE_UINT32. Its possible this might be one of the source of the issue. Let me know if any further clarification is required. Basically I am getting garbage when I read it back in. Appreciate suggestions from any HDF5 experts out there. Thanks for your time.


Solution

  • The solution lay in changing the write function to accept file and memory data types:

    bool write(const my_container<unsigned int>& p, const H5::DataType& file_datatype, const H5::DataType& mem_datatype, const std::string& datapath) const {
    
        try {
            const hsize_t h5size[1] = { p.size() };
            const H5::DataSpace h5space(1, h5size);
    
            const H5::DataSet h5set = fileptr_->createDataSet(datapath, file_datatype, h5space);
            const void* data = &(*p.begin());
            h5set.write(data, mem_datatype);
        } 
        catch( H5::Exception& ) {
            // Handle exception
            return false;
        }
    
        return true;
    }
    

    Then the rest worked as expected - read function is essentially unchanged; was able to do the same even for containers of pairs of unsigned ints and so on. HTH.