在oozie shell操作中运行pig

q43xntqr 于 2021-05-29 发布在 Hadoop

关注(0)|答案(1)|浏览(300)

我创建了一个简单的pigscript，它加载10条记录并存储在一个表中。
当我使用oozie shell操作调用这个pig（存储在hdfs中）文件时，得到如下结果和错误：

>>> Invoking Shell command line now >>
Exit code of the Shell command 5
<<< Invocation of Shell command completed <<<
<<< Invocation of Main class completed <<<
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
Oozie Launcher failed, finishing Hadoop job gracefully

我将shell文件放在工作区的lib文件夹中，并在同一lib文件夹中添加了所有必需的jar文件。请帮我解决这个问题。

hadoop oozie apache-pig oozie-coordinator

来源：https://stackoverflow.com/questions/38633951/run-pig-in-oozie-shell-action

1条答案

按热度按时间

u59ebvdq1#

我通过以下步骤解决了这个问题：

1)Created a workflow in hue placing a pig action to invoke pigscript.
2)Generated the workflow.xml file by clicking the run button.
3)Ran the workflow.xml through commandline by adding a shell wrapper class which iterates and gives dates as input parameters.

job.properties文件：

oozie.use.system.libpath=True
security_enabled=False
dryrun=False
jobTracker=<jobtracker>
nameNode=<nameNode>
oozie.wf.application.path = /user/hue/oozie/workspaces/hue-oozie-1470122057.79/workflow.xml

shell文件：

for date in 20160101 20160102 20160103
 oozie job -oozie http://<serverip>:11000/oozie -config job.properties run

赞(0）回复(0）举报 2021-05-29

我来回答

在oozie shell操作中运行pig

1条答案

相关问题

热门标签

最新问答