NDFileHDF5: Attributes defined for datasets in the xml file do not always appear in the HDF5 file.

[MarkRivers]

This problem is seen in the test2 files.

test2_layout.xml contains the following lines:

        <dataset name="aequatorial_angle" source="ndattribute" ndattribute="sa">
            <attribute name="units"       source="constant"    value="degrees" type="string" />
        </dataset>

However, the HDF5 file contains the following (this is with recent changes to "automatic" attributes in the hdf5-bugfix branch)

     DATASET "aequatorial_angle" {
        DATATYPE  H5T_IEEE_F64LE
        DATASPACE  SIMPLE { ( 5 ) / ( 5 ) }
        DATA {
        (0): 0.001, 0, 0, 0, 0
        }
        ATTRIBUTE "NDAttrDescription" {
           DATATYPE  H5T_STRING {
                 STRSIZE 34;
                 STRPAD H5T_STR_NULLTERM;
                 CSET H5T_CSET_ASCII;
                 CTYPE H5T_C_S1;
              }
           DATASPACE  SCALAR
           DATA {
           (0): "sample azimuthal rotation, degrees"
           }
        }
        ATTRIBUTE "NDAttrName" {
           DATATYPE  H5T_STRING {
                 STRSIZE 2;
                 STRPAD H5T_STR_NULLTERM;
                 CSET H5T_CSET_ASCII;
                 CTYPE H5T_C_S1;
              }
           DATASPACE  SCALAR
           DATA {
           (0): "sa"
           }
        }
        ATTRIBUTE "NDAttrSource" {
           DATATYPE  H5T_STRING {
                 STRSIZE 8;
                 STRPAD H5T_STR_NULLTERM;
                 CSET H5T_CSET_ASCII;
                 CTYPE H5T_C_S1;
              }
           DATASPACE  SCALAR
           DATA {
           (0): "ACQ_TIME"
           }
        }
        ATTRIBUTE "NDAttrSourceType" {
           DATATYPE  H5T_STRING {
                 STRSIZE 17;
                 STRPAD H5T_STR_NULLTERM;
                 CSET H5T_CSET_ASCII;
                 CTYPE H5T_C_S1;
              }
           DATASPACE  SCALAR
           DATA {
           (0): "NDAttrSourceParam"
           }
        }
     }
     DATASET "thickness" {

Note that it contains the "automatic" attributes" like NDAttrDescription, but it does not contain the "units" attribute requested in the test2_layout.xml file.

[aglowacki]

I think I fixed the constant datasets not saving attributes, as for non constant datasets whose source=ndattribute, the attribute does not get saved because of the following reason:

in NDFileHDF5.cpp function createTree

for (it_dsets = datasets.begin(); it_dsets != datasets.end(); ++it_dsets){                                        
      hid_t new_dset = this->createDataset(new_group, it_dsets->second);                                              
      if (new_dset <= 0)  continue; // failure to create the datasets so move on to next                                                
      // Write the hdf attributes to the dataset
      this->writeHdfAttributes( new_dset,  it_dsets->second);                                                         
      // Datasets are closed after data has been written                                                              
    }          

the createDataset function looks like

 if (dset->data_source().is_src_ndattribute()) {
    // At this time we cannot create the dataset as we do not know its type yet
    return -1;
    // return this->create_dataset_metadata(group, dset);
  }

Since -1 is returned for ndattributes, the writeHdfAttributes function is never called.
Later when the ndattribute datasets are created

this->createAttributeDataset(); //create the dataset
this->writeAttributeDataset(hdf5::OnFileOpen); //write the ndattribute data to the dataset
//no function to write attributes of this dataset...

We don't save the attributes of this dataset. We will have to add this functionality.

[ulrikpedersen]

Ah, yes. Because the createDataset() is only called in createTree() at NDFileHDF5.cpp:407. Creating the virtual layout tree is mostly only done only with the information given in the layout XML file - which of course cannot contain runtime information like datatype.

For detector datasets we get the (runtime defined) datatype from the NDArray* pArray argument to openFile() in NDFileHDF5.cpp:2679. So we need to get the NDAttribute datatypes from the attribute list on that same NDArray*.

[ulrikpedersen]

Fix in hdf5-issues branch PR #85