红联Linux门户
Linux帮助

Hadoop运行成功但最后显示Redirecting to job history server..

发布时间:2016-08-01 14:59:04来源:linux网站作者:sz-lcw
运行环境:ubuntu
自己生成了个jar包,用hadoop jar xxx.jar ./input ./output,运行成功,可以输出结果,但是最后提示Redirecting to job history server...Client: Retrying connect to server:
 
具体如下:
16/06/22 21:51:23 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server  
16/06/22 21:51:24 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:25 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:26 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:27 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:28 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:29 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:30 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:31 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:32 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:33 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:34 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server  
16/06/22 21:51:35 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:36 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:37 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:38 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:39 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:40 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:41 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:42 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:43 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:44 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:44 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server  
16/06/22 21:51:45 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:46 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:47 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:48 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:49 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:50 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:51 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:52 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:53 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
16/06/22 21:51:54 INFO ipc.Client: Retrying connect to server: backup01/192.168.137.200:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)  
Exception in thread "main" java.io.IOException: java.net.ConnectException: Call From backup01/192.168.137.200 to backup01:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused  
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:338)  
at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:423)  
at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:576)  
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:326)  
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:323)  
at java.security.AccessController.doPrivileged(Native Method)  
at javax.security.auth.Subject.doAs(Subject.java:422)  
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)  
at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)  
at org.apache.hadoop.mapreduce.Job.isSuccessful(Job.java:623)  
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1330)  
at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)  
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
at java.lang.reflect.Method.invoke(Method.java:498)  
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)  
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)  
Caused by: java.net.ConnectException: Call From backup01/192.168.137.200 to backup01:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused  
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)  
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)  
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)  
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)  
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)  
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)  
at org.apache.hadoop.ipc.Client.call(Client.java:1473)  
at org.apache.hadoop.ipc.Client.call(Client.java:1400)  
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)  
at com.sun.proxy.$Proxy14.getJobReport(Unknown Source)  
at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)  
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)  
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
at java.lang.reflect.Method.invoke(Method.java:498)  
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:324)  
... 17 more  
Caused by: java.net.ConnectException: Connection refused  
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)  
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)  
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)  
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)  
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:608)  
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:706)  
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:369)  
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1522)  
at org.apache.hadoop.ipc.Client.call(Client.java:1439)  
... 25 more  
 
我们可以看到,连接job history server失败。
输入jps,看有哪些java进程:
23923 QuorumPeerMain  
65477 Jps  
28614 ResourceManager  
28456 SecondaryNameNode  
28239 NameNode
真的没有启动job history service。
 
解决方案:
启动historyserver:
sbin/mr-jobhistory-daemon.sh start historyserver
 
本文永久更新地址:http://www.linuxdiyf.com/linux/22907.html