Search code examples
hadoopmapreducehadoop2

Hadoop Map-Reduce: what you can tell about a state: mappers: 80%, reducers: 20%


on interview I was asked a question: what you can tell about a state: mappers: 80%, reducers: 20% ?

My understanding is that it's impossible to get into that state. Because before reducers can start their job, all mappers should finished running: collect all key-value pairs and group them by key.

Am I correct?


Solution

  • Reducers at 20 percent means that background threads have started to copy data from mapper output locations to reducer input locations. That's why reducers progress show as 20 percent before the mapper output reaches 100 percent.