I'm using SequenceFileInputFormat as my input for my map, where the key is text and the value is text. There are 106 files each of them is between 500 MB to 750 MB. I saw my logs and it says there that the number of splits is 290.
I want to know if there is a way to increase the number of splits, because my jobs are taking a lot of time to run. Thanks for the help.
You can increase the number of splits by reducing the maximum size for each split, using the mapreduce.input.fileinputformat.split.maxsize property. The value to set is the maximum split size in bytes.