如何从命令行获取hadoop作业的成功/失败状态?

iqxoj9l9  于 2021-06-03  发布在  Hadoop
关注(0)|答案(3)|浏览(330)

我用的是cdh4和mrv1。据我所知,没有用于检查已完成作业的“状态”的命令行工具。当我转到web控制台作业详细信息页面时,我可以看到“状态:失败”或“状态:成功”。如果我跑了 mapred job -list all 或者 mapred job -status job_201309231203_0011 ,既不表示“失败”也不表示“成功”。
我是不是错过了其他命令?

u1ehiz5o

u1ehiz5o1#

hadoop job -list all
hadoop job -status <JobID>

或者hadoop jobtracker web Jmeter 板将帮助您查找此错误或与作业相关的信息。

kd3sttzy

kd3sttzy2#

我的hadoop版本是2.5.0。这对我很有用
首先获取作业id使用

hadoop job -list

那就去拿工作证

hadoop job  -status {job_id}
fkaflof6

fkaflof63#

输出的前几行 hadoop job -list all 是:

X jobs submitted
States are:
        Running : 1     Succeded : 2    Failed : 3      Prep : 4
JobId   State   StartTime       UserName        Priority        SchedulingInfo

输出行如下所示:

job_201309171413_38136  1       1382455374980   somebody        NORMAL  0 running map tasks using 0 map slots. 0 additional slots reserved. 1 running reduce tasks using 1 reduce slots. 0 additional slots reserved.
job_201309171413_37222  2       1382430339635   somebody        NORMAL  0 running map tasks using 0 map slots. 0 additional slots reserved. 0 running reduce tasks using 0 reduce slots. 0 additional slots reserved.

第二列是 State 工作的责任。根据标题行, 1 手段 Running 以及 2 手段 Succeeded . 它不是最清晰的格式:4行标题,需要引用标题来找出状态代码的实际含义,并且无法仅获取一个作业的状态。
为特定作业解析此输出的最简单方法是:

$ job_id=job_201309171413_38136
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Running

$ job_id=job_201309171413_37222
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Succeeded

$ job_id=foobar
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Unknown

相关问题