工作会开始,但只是卡住了,什么也不做。文件和作业信息如下:1. cmd上的职位信息
PS C:\hadoop-env\hadoop-3.2.1> hadoop jar C:\hadoop-env\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.2.1.jar wordcount /input/senbad.txt /output
2023-04-15 14:03:47,011 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2023-04-15 14:03:47,856 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/shekh/.staging/job_1681547354686_0002
2023-04-15 14:03:47,960 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,095 INFO input.FileInputFormat: Total input files to process : 1
2023-04-15 14:03:48,150 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,176 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,189 INFO mapreduce.JobSubmitter: number of splits:1
2023-04-15 14:03:48,286 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,302 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1681547354686_0002
2023-04-15 14:03:48,303 INFO mapreduce.JobSubmitter: Executing with tokens: []
2023-04-15 14:03:48,498 INFO conf.Configuration: resource-types.xml not found
2023-04-15 14:03:48,499 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2023-04-15 14:03:48,969 INFO impl.YarnClientImpl: Submitted application application_1681547354686_0002
2023-04-15 14:03:49,014 INFO mapreduce.Job: The url to track the job: http://LAPTOP-SEF3N6DS:8088/proxy/application_1681547354686_0002/
2023-04-15 14:03:49,015 INFO mapreduce.Job: Running job: job_1681547354686_0002
2.核心站点配置
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
3. hdfs站点配置
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>C:\hadoop-env\hadoop-3.2.1\data\dfs\namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>C:\hadoop-env\hadoop-3.2.1\data\dfs\datanode</value>
</property>
</configuration>
4. mapred站点配置
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
5. yarn站点配置
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
我在所有xml文件所在的路径中创建了resource-types.xml文件,并在此链接中给出了答案
How do I fix a "resouce-types.xml" error?
资源类型配置
<configuration>
<resources>
<resourceType name="GPU" units="NONE">
<schedulerInclude>true</schedulerInclude>
<yarnInclude>true</yarnInclude>
</resourceType>
<resourceType name="FPGA" units="NONE">
<schedulerInclude>true</schedulerInclude>
<yarnInclude>true</yarnInclude>
</resourceType>
</resources>
</configuration>
我的朋友没有这个资源类型的文件能够运行的工作罚款,但我有这个问题。
1条答案
按热度按时间w7t8yxp51#
有同样的错误,这帮助我解决了我的错误.在我的例子中,来自http://localhost:8088/cluster/nodes/unhealthy的健康报告指出了一个存在权限问题的目录。省了我很多麻烦。
这是原始溶液https://stackoverflow.com/a/64079458/14028754