Search code examples
hadoopmapreducemapr

How to prevent a Hadoop job to fail when directory is empty?

I have a job that fails when there is no files in the input directory. The exception i get is the following:

org.apache.hadoop.mapreduce.lib.input.InvalidInputException:Input Pattern maprfs:/profile/*

I know this exception is coming from the FileInputFormat class. What would be the best approach for this problem. Would overwriting the InputFormat Class be the only solution?

Thank you

Solution

    1. Put an empty file in your input path.

    2. At the beginning of your driver list your Input path(S). If there isn’t a file, then exit without submit the job.