Search code examples
hadoophdfsls

listing file names based on a pattern in hdfs

I am searching for a particular file pattern in hdfs. My requirement is that , I need to search for a filename with two characters that exists at a particular position. For Ex:

order_items_20181110_transactions.dat
order_items_20181211_transactions.dat
order_items_20181312_transactions.dat

I need to select the second file which contains 2018 and 11 just two positions after 2018

I mean is there a way to search which is as follows

hdfs dfs ls order_items_2018..11*

the two dots specifies any character which is unknown.

Solution

  • You can use the HDFS command like the below,

    hdfs dfs -ls order_items_2018??11*

    This command will list the file order_items_20181211_transactions.dat since this is the only matching result among the given three files. The question mark ? acts as a wildcard here and considers all the unknown characters at that positions. It helps to get the files of the desired pattern.

    Hope this helps.