Hadoop Streaming - Too many input params

Question

I am running a map reduce job via hadoop streaming but it gives me an error "too many params". The problem lies in the input where I am passing more than 1500 directories in input argument. I cannot reduce the number of input files.

Any suggestions.

Answer 1

You can always use wildcard to select many files without need to explicitly list them.