map becomes 100% and then drops to 0%?… here is a solution to the problem.
map becomes 100% and then drops to 0%?
I’m new to MR, but it doesn’t seem like this should happen. Here is the output:
11/12/07 13:24:42 INFO mapred. JobClient: Running job: job_201112071322_0001
11/12/07 13:24:43 INFO mapred. JobClient: map 0% reduce 0%
11/12/07 13:24:57 INFO mapred. JobClient: map 100% reduce 0%
11/12/07 13:41:48 INFO mapred. JobClient: map 100% reduce 16%
11/12/07 13:42:18 INFO mapred. JobClient: map 100% reduce 33%
11/12/07 13:59:53 INFO mapred. JobClient: map 100% reduce 0%
11/12/07 14:00:04 INFO mapred. JobClient: map 50% reduce 0%
11/12/07 14:00:24 INFO mapred. JobClient: map 0% reduce 0%
Pay attention to the time. It also just sat there, not finished and without mistakes. I guess the bug will be in the reducer, so I’ll include it; If other code is suspicious, I’ll post it, just ask.
Code: http://pastebin.com/uuL1Cgx4
Thanks a lot for any help!
Solution
I had a similar issue a few days ago. Once I reach 100%, my mapper drops to 0%.
In the end, I found that the data type definitions for the map output and the reduce input are different. So the copy/shuffle/merge data flow option between map and reduce will not work, even if hadoop JobTracker retry.
Hope it works for you.