oozie-无法识别namenode的“maprfs://”

pwuypxnk  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(287)

我在自定义mapr沙盒中运行oozie。以下是我的job.properties文件信息：

nameNode=maprfs:///
jobTracker=maprfs:///
queueName=default
examplesRoot=examples

oozie.wf.application.path=/user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml

以下是我的workflow.xml文件以供参考：

<action name="stdev-calc">
    <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>"${nameNode}"</name-node>
        <prepare>
            <delete path="/user/user01/lab2/exercise4/OUT"/>
        </prepare>
        <configuration>
            <property>
                <name>mapred.mapper.class</name>
                <value>Stdev.StdevMapper</value>
            </property>
            <property>
                <name>mapred.reducer.class</name>
                <value>Stdev.StdevReducer</value>
            </property>
            <property>
                <name>mapred.input.dir</name>
                <value>/user/user01/lab2/exercise4/DATA</value>
            </property>
            <property>
                <name>mapred.output.dir</name>
                <value>/user/user01/lab2/exercise4/OUT</value>
            </property>
        </configuration>
    </map-reduce>

    <ok to="end"/>
    <error to="fail"/>
</action>

<kill name="fail">
    <message>Exercise 4 failed with error message: [${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>

<end name="end"/>

当我尝试运行我的作业时，我的作业信息中会显示以下内容：

Job ID : 0000004-151018210741238-oozie-mapr-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : aggregator-wf
App Path      : /user/user01/lab2/exercise4/zayd_hammoudeh_lab2_exercise4/workflow.xml
Status        : KILLED
Run           : 0
User          : user01
Group         : -
Created       : 2015-10-19 04:42 GMT
Started       : 2015-10-19 04:42 GMT
Last Modified : 2015-10-19 04:42 GMT
Ended         : 2015-10-19 04:42 GMT
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext ID                 Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@:start:                                  OK        -                      OK         -
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@stdev-calc                               ERROR     -                      ERROR      IllegalArgumentException
------------------------------------------------------------------------------------------------------------------------------------
0000004-151018210741238-oozie-mapr-W@fail                                     OK        -                      OK         E0729
------------------------------------------------------------------------------------------------------------------------------------

我的日志显示错误为：“[illegalargumentexception:索引0处方案名称中的非法字符：”maprfs://“]”作为参考，下面是我的日志：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/oozie/oozie-4.1.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
2015-10-18 21:42:06,368  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] Start action [0000004-151018210741238-oozie-mapr-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,369  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action status=DONE
2015-10-18 21:42:06,372  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@:start:] [***0000004-151018210741238-oozie-mapr-W@:start:***]Action updated in DB!
2015-10-18 21:42:06,413  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Start action [0000004-151018210741238-oozie-mapr-W@stdev-calc] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,468  WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Error starting action [stdev-calc]. ErrorType [ERROR], ErrorCode [IllegalArgumentException], Message [IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"]
org.apache.oozie.action.ActionExecutorException: IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
        at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:401)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:532)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:666)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:884)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
        at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
        at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
        at org.apache.oozie.command.XCommand.call(XCommand.java:281)
        at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
        at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
        at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Illegal character in scheme name at index 0: "maprfs:///"
        at java.net.URI.create(URI.java:859)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:310)
        at org.apache.oozie.util.JobUtils.addFileToClassPath(JobUtils.java:151)
        at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:539)
        at org.apache.oozie.service.HadoopAccessorService$3.run(HadoopAccessorService.java:536)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
        at org.apache.oozie.service.HadoopAccessorService.addFileToClassPath(HadoopAccessorService.java:536)
        at org.apache.oozie.action.hadoop.JavaActionExecutor.addToCache(JavaActionExecutor.java:512)
        ... 12 more
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: "maprfs:///"
        at java.net.URI$Parser.fail(URI.java:2829)
        at java.net.URI$Parser.checkChars(URI.java:3002)
        at java.net.URI$Parser.checkChar(URI.java:3012)
        at java.net.URI$Parser.parse(URI.java:3028)
        at java.net.URI.<init>(URI.java:595)
        at java.net.URI.create(URI.java:857)
        ... 23 more
2015-10-18 21:42:06,468  WARN ActionStartXCommand:544 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] Setting Action Status to [DONE]
2015-10-18 21:42:06,494  INFO ActionEndXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@stdev-calc] ERROR is considered as FAILED for SLA
2015-10-18 21:42:06,518  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] Start action [0000004-151018210741238-oozie-mapr-W@fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-10-18 21:42:06,518  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action status=DONE
2015-10-18 21:42:06,518  INFO ActionStartXCommand:541 - SERVER[mapr1node] USER[user01] GROUP[-] TOKEN[] APP[aggregator-wf] JOB[0000004-151018210741238-oozie-mapr-W] ACTION[0000004-151018210741238-oozie-mapr-W@fail] [***0000004-151018210741238-oozie-mapr-W@fail***]Action updated in DB!

"
我想尽一切办法找出问题的原因，但都没有成功。maproozie包附带的示例建议对namenode和jobtracker使用“maprfs:///”。任何帮助都将不胜感激。

hadoopooziemapr

1条答案

按热度按时间
0kjbasz6

0kjbasz61#

我之前遗漏的问题是，当我在name节点上得到错误时，它不是由于“job.properties”文件的内容造成的。相反，它来自“workflow.xml”文件。我写错了：

<name-node>"${nameNode}"</name-node>

oozie不处理引号，如果删除引号，问题就消失了。最后的关键教训是，当您看到不容易解释的oozie错误时，要小心检查workflow.xml和job.properties文件。

赞(0）分享  回复(0）举报  2021-05-30
我来回答

相关问题

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com