我在执行以下命令时遇到了一个奇怪的错误：
hadoop jar /usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.5.1.jar -file ./methratio.py -file './genome.fa' -mapper './methratio.py -r -g ' -input /TextLab/sravisha_test/SamFiles/test_bam -output ./outfile错误：
15/01/28 18:56:29 ERROR streaming.StreamJob: Error Launching job : org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request, requested memory < 0, or requested memory > max configured, requestedMemory=3072maxMemory=2583 at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtil s.validateResourceRequest(SchedulerUtils.java:196) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateResourceRequest(RMAppManager.java:385) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:345) at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:271)
at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitA pplication(ClientRMService.java:538)
at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolP BServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:188)
at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:323)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)有人能告诉我这里怎么了吗？
1条答案
按热度按时间klh5stk11#
该错误指出yarn环境中的节点内存不足。可能是节点计算机/vm没有足够的内存，或者
yarn.nodemanager.resource.memory-mb价值yarn-site.xml设置为低于3072的值，但流应用程序需要更多内存。