使用webhdfs配置apacheknox网关的问题

rqmkfv5c  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(634)

我已经使用apacheambari安装了apacheknox。按照下面的链接使用knox配置webhdfs。knox安装在 address1 . https://knox.apache.org/books/knox-0-13-0/user-guide.html#webhdfs
使用curl命令调用webhdfs时 curl -i -k -u guest:guest-password -X GET 'https://address1:8443/gateway/Andromd/webhdfs/v1/?op=LISTSTATUS' 它抛出以下错误

<title> 404 Not Found</title>
<p> Problem accessing /gateway//Andromd/webhdfs/v1/build-version.Reason:
<pre> Not Found </pre></p>

andromd是集群名称,并将其添加到 {GATEWAY_HOME}/conf/topologies/Andromd.xml . 配置如下。

<service>
    <role>NAMENODE</role>
    <url>hdfs://address2:8020</url>
</service>
<service>
    <role>WEBHDFS</role>
    <url>http://address2:50070/webhdfs</url>
</service

的内容 /etc/hadoop/conf/hdfs-site.xml 具体如下。

<property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
</property>
<property>
    <name>dfs.namenode.rpc-address</name>
    <value>address2:8020</value>
</property>
<property>
    <name>dfs.namenode.http-address</name>
    <value>address2:50070</value>
</property>
<property>
    <name>dfs.https.namenode.https-address</name>
    <value>address2:50470</value>
</property>

一定要让我知道,是否有任何东西从我这边关于配置缺失。
gateway.log内容:

2017-10-13 18:24:24,586 ERROR digester3.Digester (Digester.java:parse(1652)) - An error occurred while parsing XML from '(already loaded from stream)', see nested exceptions
org.xml.sax.SAXParseException; lineNumber: 88; columnNumber: 18; The content of elements must consist of well-formed character data or markup.
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1239)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
        at org.apache.commons.digester3.Digester.parse(Digester.java:1642)
        at org.apache.commons.digester3.Digester.parse(Digester.java:1701)
        at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.loadTopologyAttempt(DefaultTopologyService.java:124)
        at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.loadTopology(DefaultTopologyService.java:100)
        at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.loadTopologies(DefaultTopologyService.java:233)
        at org.apache.hadoop.gateway.services.topology.impl.DefaultTopologyService.reloadTopologies(DefaultTopologyService.java:318)
        at org.apache.hadoop.gateway.GatewayServer.start(GatewayServer.java:312)
        at org.apache.hadoop.gateway.GatewayServer.startGateway(GatewayServer.java:231)
        at org.apache.hadoop.gateway.GatewayServer.main(GatewayServer.java:114)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
        at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
        at org.apache.hadoop.gateway.launcher.Command.run(Command.java:101)
        at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
        at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
2017-10-13 18:24:24,587 ERROR hadoop.gateway (DefaultTopologyService.java:loadTopologies(250)) - Failed to load topology /usr/hdp/2.4.2.0-258/knox/bin/../conf/topologies/Andromeda.xml: org.xml.sax.SAXParseException; lineNumber: 88; columnNumber: 18; The content of elements must consist of well-formed character data or markup.
2017-10-13 18:24:24,588 INFO  hadoop.gateway (GatewayServer.java:handleCreateDeployment(450)) - Loading topology admin from /usr/hdp/2.4.2.0-258/knox/bin/../data/deployments/admin.war.15f15c171c0
2017-10-13 18:24:24,793 INFO  hadoop.gateway (GatewayServer.java:start(315)) - Monitoring topologies in directory: /usr/hdp/2.4.2.0-258/knox/bin/../conf/topologies
2017-10-13 18:24:24,795 INFO  hadoop.gateway (GatewayServer.java:startGateway(232)) - Started gateway on port 8,443.

添加到上面的查询中,需要在 Andromd.xml ,通过替换 sandbox.hortonworks.com 我们的集群特定信息。

<property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
</property>
<property>
    <name>dfs.namenode.rpc-address</name>
    <value>sandbox.hortonworks.com:8020</value>
</property>
<property>
    <name>dfs.namenode.http-address</name>
    <value>sandbox.hortonworks.com:50070</value>
</property>
<property>
    <name>dfs.https.namenode.https-address</name>
    <value>sandbox.hortonworks.com:50470</value>
</property>
hsvhsicv

hsvhsicv1#

2017-10-13 18:24:24,587 ERROR hadoop.gateway (DefaultTopologyService.java:loadTopologies(250)) - Failed to load topology /usr/hdp/2.4.2.0-258/knox/bin/../conf/topologies/Andromeda.xml: org.xml.sax.SAXParseException; lineNumber: 88; columnNumber: 18; The content of elements must consist of well-formed character data or markup.

看这一行,看起来您的拓扑文件andromeda.xml在行号88处的格式不好(缺少结束标记或类似的标记)。
这就是为什么您得到404,因为没有部署拓扑。修复拓扑文件后检查日志,确保没有启动错误。

相关问题