找不到“resource-types.xml”,这显示当我尝试在win10上的Hadoop上运行wordcount作业时

lx0bsm1f  于 8个月前  发布在  Hadoop
关注(0)|答案(1)|浏览(205)

工作会开始,但只是卡住了,什么也不做。文件和作业信息如下:1. cmd上的职位信息

PS C:\hadoop-env\hadoop-3.2.1> hadoop jar C:\hadoop-env\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.2.1.jar wordcount /input/senbad.txt /output
2023-04-15 14:03:47,011 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2023-04-15 14:03:47,856 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/shekh/.staging/job_1681547354686_0002
2023-04-15 14:03:47,960 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,095 INFO input.FileInputFormat: Total input files to process : 1
2023-04-15 14:03:48,150 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,176 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,189 INFO mapreduce.JobSubmitter: number of splits:1
2023-04-15 14:03:48,286 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,302 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1681547354686_0002
2023-04-15 14:03:48,303 INFO mapreduce.JobSubmitter: Executing with tokens: []
2023-04-15 14:03:48,498 INFO conf.Configuration: resource-types.xml not found
2023-04-15 14:03:48,499 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2023-04-15 14:03:48,969 INFO impl.YarnClientImpl: Submitted application application_1681547354686_0002
2023-04-15 14:03:49,014 INFO mapreduce.Job: The url to track the job: http://LAPTOP-SEF3N6DS:8088/proxy/application_1681547354686_0002/
2023-04-15 14:03:49,015 INFO mapreduce.Job: Running job: job_1681547354686_0002

2.核心站点配置

<configuration>
   <property>
       <name>fs.defaultFS</name>
       <value>hdfs://localhost:9000</value>
   </property>
</configuration>

3. hdfs站点配置

<configuration>
<property>
       <name>dfs.replication</name>
       <value>1</value>
   </property>
   <property>
       <name>dfs.namenode.name.dir</name>
       <value>C:\hadoop-env\hadoop-3.2.1\data\dfs\namenode</value>
   </property>
   <property>
       <name>dfs.datanode.data.dir</name>
       <value>C:\hadoop-env\hadoop-3.2.1\data\dfs\datanode</value>
   </property>

</configuration>

4. mapred站点配置

<configuration>
   <property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
   </property>
</configuration>

5. yarn站点配置

<configuration>
   <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
   </property>
   <property>
          <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>  
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
   </property>
</configuration>

我在所有xml文件所在的路径中创建了resource-types.xml文件,并在此链接中给出了答案
How do I fix a "resouce-types.xml" error?

资源类型配置

<configuration>
  <resources>
    <resourceType name="GPU" units="NONE">
      <schedulerInclude>true</schedulerInclude>
      <yarnInclude>true</yarnInclude>
    </resourceType>
    <resourceType name="FPGA" units="NONE">
      <schedulerInclude>true</schedulerInclude>
      <yarnInclude>true</yarnInclude>
    </resourceType>
  </resources>
</configuration>

我的朋友没有这个资源类型的文件能够运行的工作罚款,但我有这个问题。

w7t8yxp5

w7t8yxp51#

有同样的错误,这帮助我解决了我的错误.在我的例子中,来自http://localhost:8088/cluster/nodes/unhealthy的健康报告指出了一个存在权限问题的目录。省了我很多麻烦。
这是原始溶液https://stackoverflow.com/a/64079458/14028754

相关问题