Hive作业正在处理,处理后某个时间停止

ckocjqey  于 2021-06-26  发布在  Hive
关注(0)|答案(1)|浏览(312)

我在一台独立的机器上运行hive。hadoop正在伪分布式模式下运行。我正在运行连接两个表的配置单元查询(一个表有7m条记录,另一个有51m条记录,每个记录包含8列)。经过一段时间的处理后,mapper达到零百分比,然后偶尔继续打印零。你能帮我解决这个问题吗。
参考以下日志。

2016-04-12 22:52:58,469 Stage-1 map = 71%,  reduce = 1%
2016-04-12 22:53:00,517 Stage-1 map = 72%,  reduce = 1%
2016-04-12 22:53:02,560 Stage-1 map = 73%,  reduce = 1%
2016-04-12 22:53:09,740 Stage-1 map = 74%,  reduce = 1%
2016-04-12 22:53:11,796 Stage-1 map = 75%,  reduce = 1%
2016-04-12 22:53:13,842 Stage-1 map = 76%,  reduce = 1%
2016-04-12 22:53:21,037 Stage-1 map = 77%,  reduce = 1%
2016-04-12 22:53:24,114 Stage-1 map = 78%,  reduce = 1%
2016-04-12 22:53:26,156 Stage-1 map = 79%,  reduce = 1%
2016-04-12 22:53:35,433 Stage-1 map = 81%,  reduce = 1%
2016-04-12 22:53:38,507 Stage-1 map = 82%,  reduce = 1%
2016-04-12 22:53:45,725 Stage-1 map = 82%,  reduce = 0%
2016-04-12 22:53:49,925 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:54:50,236 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:55:50,546 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:56:50,863 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:57:51,128 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:58:51,352 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:59:51,612 Stage-1 map = 0%,  reduce = 0%
2016-04-12 23:00:51,886 Stage-1 map = 0%,  reduce = 0%
2016-04-12 23:01:52,131 Stage-1 map = 0%,  reduce = 0%

我在追踪器里确认了状态。状态显示两次尝试,一次尝试失败,诊断消息如下。

AM Container for appattempt_1460481465127_0001_000001 exited with exitCode: -100
For more detailed output, check application tracking page:http://localhost:8088/cluster/app/application_1460481465127_0001Then, click on links to logs of each attempt.
Diagnostics: Container released on a *lost* nodeFailing this attempt

提前谢谢。

frebpwbc

frebpwbc1#

这个问题似乎是由map端的堆空间引起的。
尝试通过执行以下操作来增加Map任务堆大小:
mapred-site.xml (尝试调整以下值以与您的用例匹配):

<!-- for mappers -->
<property>
   <name>mapreduce.map.memory.mb</name>
   <value>4096</value>
</property>
<!-- For reduces -->
<property>
   <name>mapreduce.reduce.memory.mb</name>
   <value>8192</value>
</property>

相关问题