Why is the hadoop output file part-r-00000 empty… here is a solution to the problem.
Why is the hadoop output file part-r-00000 empty
My MR logs are:
[root@sicongli hadoop-2.4.1]# hadoop jar flowcount.jar
cn.itheima.bigdata.hadoop.mr.flowcount.FlowCount /data/join.txt /out
16/04/13 23:32:20 WARN util. NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
16/04/13 23:32:22 INFO client. RMProxy: Connecting to ResourceManager at sicongli/192.168.218.111:8032
16/04/13 23:32:28 WARN mapreduce. JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/04/13 23:32:35 INFO input. FileInputFormat: Total input paths to process : 1
16/04/13 23:32:38 INFO mapreduce. JobSubmitter: number of splits:1
16/04/13 23:32:41 INFO mapreduce. JobSubmitter: Submitting tokens for job: job_1460601112521_0002
16/04/13 23:32:47 INFO impl. YarnClientImpl: Submitted application application_1460601112521_0002
16/04/13 23:32:47 INFO mapreduce. Job: The url to track the job: http://sicongli:8088/proxy/application_1460601112521_0002/
16/04/13 23:32:47 INFO mapreduce. Job: Running job: job_1460601112521_0002
16/04/13 23:35:20 INFO mapreduce. Job: Job job_1460601112521_0002 running in uber mode : false
16/04/13 23:35:28 INFO mapreduce. Job: map 0% reduce 0%
16/04/13 23:36:47 INFO mapreduce. Job: map 100% reduce 0%
16/04/13 23:37:25 INFO mapreduce. Job: map 100% reduce 100%
16/04/13 23:37:48 INFO mapreduce. Job: Job job_1460601112521_0002 completed successfully
16/04/13 23:38:16 INFO mapreduce. Job: Counters: 49
File System Counters
FILE: Number of bytes read=6
FILE: Number of bytes written=186579
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=399
HDFS: Number of bytes written=0
HDFS: Number of read operations=6
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=17296
Total time spent by all reduces in occupied slots (ms)=36727
Total time spent by all map tasks (ms)=17296
Total time spent by all reduce tasks (ms)=36727
Total vcore-seconds taken by all map tasks=17296
Total vcore-seconds taken by all reduce tasks=36727
Total megabyte-seconds taken by all map tasks=17711104
Total megabyte-seconds taken by all reduce tasks=37608448
Map-Reduce Framework
Map input records=23
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=99
Combine input records=0
Combine output records=0
Reduce input groups=0
Reduce shuffle bytes=6
Reduce input records=0
Reduce output records=0
Spilled Records=0
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=217
CPU time spent (ms)=1150
Physical memory (bytes) snapshot=277962752
Virtual memory (bytes) snapshot=1689296896
Total committed heap usage (bytes)=127127552
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=300
File Output Format Counters
Bytes Written=0
16/04/13 23:38:18 INFO ipc. Client: Retrying connect to server: sicongli/192.168.218.111:49806. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
16/04/13 23:38:19 INFO ipc. Client: Retrying connect to server: sicongli/192.168.218.111:49806. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
16/04/13 23:38:20 INFO ipc. Client: Retrying connect to server: sicongli/192.168.218.111:49806. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
16/04/13 23:38:23 INFO mapred. ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
The output is:
[root@sicongli ~]# hadoop fs -ls /out
16/04/14 00:00:38 WARN util. NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r-- 3 root supergroup 0 2016-04-13 23:37 /out/_SUCCESS
-rw-r--r-- 3 root supergroup 0 2016-04-13 23:37 /out/part-r-00000
I have two questions:
One: Why is the output file part-r-0000 empty
tow : Why does the warning appear: INFO ipc. Client: Retrying connect to server: sicongli/192.168.218.111:49806。 Already tried 2 times; The retry strategy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS).
Solution
Issue 1 – Read counters:
Map input records=23
Map output records=0
Part-r-00000 is empty because there is nothing in your map task. If you add the code for the map task to your post, we may tell you why.
Question 2 – Read the answer questions for this and they may help you.