Search code examples
pandasstringpattern-matchingmulti-index

How to find a partial string match within a pandas.MultiIndex


I am looking to have a partial string match on "ne" and "tw".

Basic Setup:

arrays = [np.array(['bar', 'bar', 'baz', 'baz', 'foo', 'foo', 'qux', 'qux']),
         np.array(['one', 'two', 'three', 'four', 'five', 'six', 'nine', 'none'])]

s = pd.Series(np.random.randn(8), index=arrays) df = pd.DataFrame(np.random.randn(8, 4), index=arrays)

df Out[2]:
                  0         1         2         3
bar one    1.238627  0.027646 -1.707589 -0.769593
    two    0.144709  0.087773  0.725266 -0.463602
baz three  2.098599  0.551828 -0.129251  1.150297
    four   0.784710  1.957488 -0.919756 -0.291112
foo five   0.578707  0.292793  0.129004 -0.704882
    six   -0.539508 -0.301554 -0.350157  0.018169
qux nine   0.404407 -1.226800 -1.463461 -2.569753
    none   0.774964  0.204157 -0.695053 -1.161872

to get:

    Out[3]: 
                 0         1         2         3
bar one  -0.759341  0.979908  0.423735  0.224613
    two   1.224353 -0.287837  1.020571  2.633446
qux nine  0.888379  0.773314  1.507129 -0.279791
    none -0.967281 -1.239551  0.609369 -0.725862

In a single index I would simply do:

df[df.index.str.contains("ne"))]

For multiple partial string matches:

df[df.index.str.contains('|'.join(["ne","tw"))]

What's the best option for selecting partial-string matches? Respectfully, why isn't there much support for MultiIndex as other parts of pandas?

Thanks!


Solution

  • You can select a specific index out of a MultiIndex, then run .str.contains on that:

    # df.index.get_level_values(1) returns an pd.Index object
    df.loc[df.index.get_level_values(1).str.contains("ne|tw")]
    
                     0         1         2         3
    bar one   0.513132 -0.646786 -1.687423  2.614390
        two  -1.070990  1.618638 -1.485864 -0.813031
    qux nine -0.438507 -0.830141  0.009474  0.206083
        none -0.811970  0.342299 -0.165243 -1.482466