我已经在多节点集群(1个namenode和4个datanode)上成功运行了hadoop2.7.1。但是,当我运行mapreduce作业(来自hadoop网站的wordcount示例)时,它总是停留在这一点上。
[~@~ hadoop-2.7.1]$ bin/hadoop jar WordCount.jar WordCount /user/inputdata/ /user/outputdata
15/09/30 17:54:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/09/30 17:54:57 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/09/30 17:54:58 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
15/09/30 17:54:59 INFO input.FileInputFormat: Total input paths to process : 1
15/09/30 17:55:00 INFO mapreduce.JobSubmitter: number of splits:1
15/09/30 17:55:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1443606819488_0002
15/09/30 17:55:00 INFO impl.YarnClientImpl: Submitted application application_1443606819488_0002
15/09/30 17:55:00 INFO mapreduce.Job: The url to track the job: http://~~~~:8088/proxy/application_1443606819488_0002/
15/09/30 17:55:00 INFO mapreduce.Job: Running job: job_1443606819488_0002
我必须为指定内存吗 yarn
?
注意:datanode硬件非常旧(每个都有1gb ram)。
谢谢你的帮助。谢谢您。
1条答案
按热度按时间mznpcxlj1#
数据节点内存(1gb)非常有限,至少需要准备一个容器来运行mapper/reducer/am。
您可以尝试在中降低以下容器内存分配值
yarn-site.xml
以获得在其上创建的容器。还要尽量减少作业配置中的以下属性值,