Search code examples
pandaspytableshdfstore

Check whether a PyTables node in a pandas HDFStore is tabular


Is there a preferred way to check whether a PyTables node in a pandas HDFStore is tabular? This works, but NoSuchNodeError doesn't seem like part of the API, so maybe I should not rely on it.

In [34]: from tables.table import NoSuchNodeError

In [35]: def is_tabular(store, key):
    try:
        store.get_node(key).table
    except NoSuchNodeError:
        return False
    return True
   ....: 

In [36]: is_tabular(store, 'first_600')
Out[36]: False

In [37]: is_tabular(store, 'features')
Out[37]: True

Solution

  • You could do something like this. The pandas_type, table_type meta-data will be present in the pytables attribute _v_attrs at the top-level of the node.

    In [28]: store = pd.HDFStore('test.h5',mode='w')
    
    In [29]: store.append('df',DataFrame(np.random.randn(10,2),columns=list('AB')))
    
    In [30]: store
    Out[30]: 
    <class 'pandas.io.pytables.HDFStore'>
    File path: test.h5
    /df            frame_table  (typ->appendable,nrows->10,ncols->2,indexers->[index])
    
    In [31]: store._handle.root.df._v_attrs
    Out[31]: 
    /df._v_attrs (AttributeSet), 14 attributes:
       [CLASS := 'GROUP',
        TITLE := u'',
        VERSION := '1.0',
        data_columns := [],
        encoding := None,
        index_cols := [(0, 'index')],
        info := {1: {'type': 'Index', 'names': [None]}, 'index': {}},
        levels := 1,
        nan_rep := 'nan',
        non_index_axes := [(1, ['A', 'B'])],
        pandas_type := 'frame_table',
        pandas_version := '0.10.1',
        table_type := 'appendable_frame',
        values_cols := ['values_block_0']]
    
    In [33]: getattr(getattr(getattr(store._handle.root,'df',None),'_v_attrs',None),'pandas_type',None)
    Out[33]: 'frame_table'
    
    In [34]: store.close()
    
    In [35]: