Search code examples
hadoopmapreducecascading

Deleting outputPath in cascading


We generally delete the output path in mapreduce by using the following code

outputPath.getFileSystem(conf).delete(outputPath,true);

My question is, can we do the same in cascading? If yes please suggest me the steps for doing so.
Thankyou:)

Solution

  • If the intent is to reuse the output directory for next run of mapreduce then set the property of your sink Tap to SinkMode.REPLACE. It deletes the hdfs output DIR and mapreduce jobs will not fails between re runs.

    Tap sink = new Hfs(<your scheme>,<your output hdfs dir>,SinkMode.REPLACE)