作业失败,因为任务失败failedmaps:1 failedreduces:0使用hadoop和giraph时出现异常

ma8fv8wu  于 2021-05-27  发布在  Hadoop
关注(0)|答案(0)|浏览(1057)

我正在使用一个虚拟机,拥有一个由1个主节点和3个节点组成的集群,我安装了hadoop并将其复制到所有节点,并在主节点中部署giraph,但我不知道是否也应该将giraph文件夹复制到所有节点!!,我使用的是真正的数据集wikitalk,我的命令行是
$hadoop\u home/bin/hadoop jar$giraph\u home/giraph examples/target/giraph-examples-1.3.0-snapshot-for-hadoop-2.5.1-jar-with-dependencies.jar org.apache.giraph.giraphrunner org.apache.giraph.examples.simplepagerankcomputation-vif org.apache.giraph.io.formats.jsonlongtublefloatdoublevertexinputformat-vip/user/hduser/input/wikitalk.txt-voforg.apache.giraph.io.formats.idwithvaluetextoutputformat-op/user/hduser/output/wikitalk11\u output-w 4
但我发现了这个例外

20/09/22 23:14:28 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 5 mappers
20/09/22 23:14:50 INFO mapreduce.Job: Running job: job_1600808538899_0008
20/09/22 23:14:50 INFO mapreduce.Job: Job job_1600808538899_0008 running in uber mode : false
20/09/22 23:14:50 INFO mapreduce.Job:  map 100% reduce 0%
20/09/22 23:14:50 INFO mapreduce.Job: Job job_1600808538899_0008 failed with state FAILED due to: Task failed task_1600808538899_0008_m_000002
Job failed as tasks failed. failedMaps:1 failedReduces:0

20/09/22 23:14:50 INFO mapreduce.Job: Counters: 12
    Job Counters 
        Failed map tasks=1
        Killed map tasks=4
        Launched map tasks=5
        Other local map tasks=5
        Total time spent by all maps in occupied slots (ms)=65129
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=65129
        Total vcore-seconds taken by all map tasks=65129
        Total megabyte-seconds taken by all map tasks=66692096
    Map-Reduce Framework
        CPU time spent (ms)=0
        Physical memory (bytes) snapshot=0
        Virtual memory (bytes) snapshot=0

所以我不知道这个例外的原因是什么?!!

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题