hive不支持hdfs联合

epfja78i  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(361)

配置单元版本是apache hive-2.1.0 tez版本是apache tez-0.8.4 我准备好了 tez.lib.uris=viewfs://nsX/nameservice/hive/lib/tez-0.8.4-zdh7.1.1-SNAPSHOT-minimal.tar.gz 当我在hive上运行一个tez作业时,总是出现问题

2017-03-23T15:26:24,255 INFO  client.TezClientUtils (:()) - Using   tez.lib.uris value from configuration: viewfs://nsX/nameservice/hive/lib/tez-0.8.4-zdh7.1.1-SNAPSHOT-minimal.tar.gz
  2017-03-23T15:26:24,255 INFO  client.TezClientUtils (:()) - Using tez.lib.uris.classpath value from configuration: $HADOOP_CONF_DIR,/opt/ZDH/parcels/lib/hadoop/*,/opt/ZDH/parcels/lib/hadoop/lib/*,/opt/ZDH/parcels/lib/hadoop-hdfs/*,/opt/ZDH/parcels/lib/hadoop-hdfs/lib/*,/opt/ZDH/parcels/lib/hadoop-yarn/*,/opt/ZDH/parcels/lib/hadoop-yarn/lib/*,/opt/ZDH/parcels/lib/hadoop-mapreduce/*,/opt/ZDH/parcels/lib/hadoop-mapreduce/lib/*,/opt/ZDH/parcels/lib/tez/*,/opt/ZDH/parcels/lib/tez/lib/*
  2017-03-23T15:26:24,260 ERROR exec.Task (:()) - Failed to execute tez graph.
java.io.FileNotFoundException: /hive
    at org.apache.hadoop.fs.viewfs.InodeTree.resolve(InodeTree.java:402)
    at org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:377)
    at org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:1031)
    at org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:275)
    at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183)
    at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1057)
    at org.apache.tez.client.TezClient.start(TezClient.java:447)
    at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:390)
    at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:320)
    at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager$TezSessionPoolSession.openInternal(TezSessionPoolManager.java:611)
    at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:203)
    at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:277)
    at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:155)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1917)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1586)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1331)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1092)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1080)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
n3ipq98p

n3ipq98p1#

根据您的错误,这是类似于tez引擎无法找到文件配置单元。
或者,您可以做一件事,登录到hiveshell并在这里将执行引擎设置为tez,然后在hiveshell中运行查询,它们将依次在tez引擎上运行。

set hive.execution.engine=tez;

有关将tez与hadoop集成的更多信息,请参阅此链接。

相关问题