Java – map becomes 100% and then drops to 0%?

map becomes 100% and then drops to 0%?… here is a solution to the problem.

map becomes 100% and then drops to 0%?

I’m new to MR, but it doesn’t seem like this should happen. Here is the output:

 11/12/07 13:24:42 INFO mapred. JobClient: Running job: job_201112071322_0001
 11/12/07 13:24:43 INFO mapred. JobClient:  map 0% reduce 0%
 11/12/07 13:24:57 INFO mapred. JobClient:  map 100% reduce 0%
 11/12/07 13:41:48 INFO mapred. JobClient:  map 100% reduce 16%
 11/12/07 13:42:18 INFO mapred. JobClient:  map 100% reduce 33%
 11/12/07 13:59:53 INFO mapred. JobClient:  map 100% reduce 0%
 11/12/07 14:00:04 INFO mapred. JobClient:  map 50% reduce 0%
 11/12/07 14:00:24 INFO mapred. JobClient:  map 0% reduce 0%

Pay attention to the time. It also just sat there, not finished and without mistakes. I guess the bug will be in the reducer, so I’ll include it; If other code is suspicious, I’ll post it, just ask.

Code: http://pastebin.com/uuL1Cgx4

Thanks a lot for any help!

Solution

I had a similar issue a few days ago. Once I reach 100%, my mapper drops to 0%.

In the end, I found that the data type definitions for the map output and the reduce input are different. So the copy/shuffle/merge data flow option between map and reduce will not work, even if hadoop JobTracker retry.

Hope it works for you.

Related Problems and Solutions