使用kerberos安全身份验证从zookeeper读取文件的问题

odopli94  于 2021-06-09  发布在  Hbase
关注(0)|答案(0)|浏览(323)

我正在使用 Kerberos secure authentication 从中读取数据 ZooKeeper 从两台linux服务器。在一个服务器上连接到 ZooKeeper 断断续续地丢失，当我检查日志中的错误时，每次我丢失连接时都会看到几个不同的错误消息。每次我都要重新启动服务器才能连接到 ZooKeeper 再一次。在另一台服务器上，它运行良好，所以不确定在哪里检查这个问题。我在两台服务器上运行相同的代码，唯一的区别是一台运行Java1.6，另一台运行Java1.7。java版本会是一个问题吗？
下面是我的代码。。。

if (!UserGroupInformation.isSecurityEnabled()) {
      try
      {
        throw new IOException("Security is not enabled in core-site.xml");
      }
      catch (IOException e1)
      {
        e1.printStackTrace();
      }
    }
    try
    {
      String uName = conf.get("kerbose.user");
      String ktabPath = conf.get("keytabPath");
      System.out.println("Logging in1" + uName);
      UserGroupInformation.setConfiguration(conf);

      UserGroupInformation.loginUserFromKeytab(uName, ktabPath);
    }
    catch (IOException e)
    {
      e.printStackTrace();
    }

这是我导出的java变量。。

export JAVA_OPTS="-Xms1024m -Xmx2048m -Djava.security.krb5.conf=/PATH TO FILE/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false"

下面是我的hbase-site.xml。。。

<configuration>
<property>
    <name>hbase.env</name>
    <value>qat</value>
</property>
<property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
</property>
<property>
  <name>hbase.security.authorization</name>
  <value>true</value>
</property>
<property>
    <name>kerbose.user</name>
    <value>USERNAME</value>
</property>

<property>
    <name>keytabPath</name>
    <value>/PATHTO/flume.service.keytab</value>
</property>

<property>
  <name>dfs.domain.socket.path</name>
  <value>/var/lib/hadoop-hdfs/dn_socket</value>
</property>

<property>
  <name>hbase.bucketcache.ioengine</name>
  <value>offheap</value>
</property>

<property>
  <name>hbase.bucketcache.percentage.in.combinedcache</name>
  <value/>
</property>

<property>
  <name>hbase.bucketcache.size</name>
  <value>10240</value>
</property>

<property>
  <name>hbase.bulkload.staging.dir</name>
  <value>/apps/hbase/staging</value>
</property>

<property>
  <name>hbase.client.keyvalue.maxsize</name>
  <value>10485760</value>
</property>

<property>
  <name>hbase.client.retries.number</name>
  <value>35</value>
</property>

<property>
  <name>hbase.client.scanner.caching</name>
  <value>100</value>
</property>

<property>
  <name>hbase.cluster.distributed</name>
  <value>true</value>
</property>

<property>
  <name>hbase.coprocessor.abortonerror</name>
  <value>false</value>
</property>

<property>
  <name>hbase.coprocessor.master.classes</name>
  <value>org.apache.hadoop.hbase.security.access.AccessController</value>
</property>

<property>
  <name>hbase.coprocessor.region.classes</name>
  <value>org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController</value>
</property>

<property>
  <name>hbase.defaults.for.version.skip</name>
  <value>true</value>
</property>

<property>
  <name>hbase.hregion.majorcompaction</name>
  <value>604800000</value>
</property>

<property>
  <name>hbase.hregion.majorcompaction.jitter</name>
  <value>0.50</value>
</property>

<property>
  <name>hbase.hregion.max.filesize</name>
  <value>10737418240</value>
</property>

<property>
  <name>hbase.hregion.memstore.block.multiplier</name>
  <value>4</value>
</property>

<property>
  <name>hbase.hregion.memstore.flush.size</name>
  <value>134217728</value>
</property>

<property>
  <name>hbase.hregion.memstore.mslab.enabled</name>
  <value>true</value>
</property>

<property>
  <name>hbase.hstore.blockingStoreFiles</name>
  <value>10</value>
</property>

<property>
  <name>hbase.hstore.compaction.max</name>
  <value>10</value>
</property>

<property>
  <name>hbase.hstore.compactionThreshold</name>
  <value>3</value>
</property>

<property>
  <name>hbase.local.dir</name>
  <value>${hbase.tmp.dir}/local</value>
</property>

<property>
  <name>hbase.master.info.bindAddress</name>
  <value>0.0.0.0</value>
</property>

<property>
  <name>hbase.master.info.port</name>
  <value>60010</value>
</property>

<property>
  <name>hbase.master.kerberos.principal</name>
  <value>hbase/VALUE</value>
</property>

<property>
  <name>hbase.master.keytab.file</name>
  <value>/etc/security/keytabs/hbase.service.keytab</value>
</property>

<property>
  <name>hbase.master.port</name>
  <value>60000</value>
</property>

<property>
  <name>hbase.region.server.rpc.scheduler.factory.class</name>
  <value>org.apache.phoenix.hbase.index.ipc.PhoenixIndexRpcSchedulerFactory</value>
</property>

<property>
  <name>hbase.regionserver.global.memstore.lowerLimit</name>
  <value>0.38</value>
</property>

<property>
  <name>hbase.regionserver.global.memstore.size</name>
  <value>0.4</value>
</property>

<property>
  <name>hbase.regionserver.global.memstore.upperLimit</name>
  <value>0.4</value>
</property>

<property>
  <name>hbase.regionserver.handler.count</name>
  <value>60</value>
</property>

<property>
  <name>hbase.regionserver.info.port</name>
  <value>60030</value>
</property>

<property>
  <name>hbase.regionserver.kerberos.principal</name>
  <value>hbase/VALUE</value>
</property>

<property>
  <name>hbase.regionserver.keytab.file</name>
  <value>/etc/security/keytabs/hbase.service.keytab</value>
</property>

<property>
  <name>hbase.regionserver.wal.codec</name>
  <value>org.apache.hadoop.hbase.regionserver.wal.WALCellCodec</value>
</property>

<property>
  <name>hbase.replication</name>
  <value>true</value>
</property>

<property>
  <name>hbase.rootdir</name>
  <value>hdfs://SERVER/apps/hbase/data</value>
</property>

<property>
  <name>hbase.rpc.controllerfactory.class</name>
  <value/>
</property>

<property>
  <name>hbase.rpc.protection</name>
  <value>authentication</value>
</property>

<property>
  <name>hbase.rpc.timeout</name>
  <value>90000</value>
</property>

<property>
  <name>hbase.security.authentication</name>
  <value>kerberos</value>
</property>

<property>
  <name>hbase.security.authorization</name>
  <value>true</value>
</property>

<property>
  <name>hbase.superuser</name>
  <value>hbase</value>
</property>

<property>
  <name>hbase.tmp.dir</name>
  <value>/data0/hadoop/hbase</value>
</property>

<property>
  <name>hbase.zookeeper.property.clientPort</name>
  <value>2181</value>
</property>

<property>
  <name>hbase.zookeeper.quorum</name>
  <value>ZOOKEEPERSERVERS LIST</value>
</property>

<property>
  <name>hbase.zookeeper.useMulti</name>
  <value>true</value>
</property>

<property>
  <name>hfile.block.cache.size</name>
  <value>0.40</value>
</property>

<property>
  <name>phoenix.functions.allowUserDefinedFunctions</name>
  <value> </value>
</property>

<property>
  <name>phoenix.query.timeoutMs</name>
  <value>60000</value>
</property>

<property>
  <name>zookeeper.session.timeout</name>
  <value>30000</value>
</property>

<property>
  <name>zookeeper.znode.parent</name>
  <value>/hbase-secure</value>
</property>

请让我知道在哪里检查这个问题。谢谢。

编辑

这是错误。。。

2016-03-09 11:44:01,382 [http-bio-8080-exec-28] ERROR security.UserGroupInformation  - PriviledgedActionException as:USERNAME (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Attempt to obtain new INITIATE credentials failed! (null))]
2016-03-03 16:08:14,851 [http-bio-8080-exec-22] WARN  client.HConnectionManager$HConnectionImplementation  - Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:
Thu Mar 03 15:59:02 PST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@db5183, java.io.IOException: Call to SERVERIP:60020 failed on local exception: java.io.EOFException
2016-03-05 16:06:33,294 [http-bio-8080-exec-6] ERROR security.UserGroupInformation  - PriviledgedActionException as:USERNAME (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Attempt to obtain new INITIATE credentials failed! (null))]
2016-03-06 00:00:23,253 [http-bio-8080-exec-19] WARN  security.UserGroupInformation  - Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
2016-03-06 00:00:23,935 [http-bio-8080-exec-19] ERROR security.UserGroupInformation  - PriviledgedActionException as:USERNAME (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Attempt to obtain new INITIATE credentials failed! (null))]
2016-03-06 00:00:23,935 [http-bio-8080-exec-19] WARN  ipc.RpcClient  - Couldn't setup connection for USERNAME to hbase/SERVER
2016-03-06 00:00:23,936 [http-bio-8080-exec-19] ERROR security.UserGroupInformation  - PriviledgedActionException as:USERNAME (auth:KERBEROS) cause:java.io.IOException: Couldn't setup connection for USERNAME to hbase/SERVER
Sun Mar 06 07:57:19 PST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@3dd66a, java.io.IOException: Couldn't setup connection for USERNAME to hbase/SERVER
Sun Mar 06 07:57:46 PST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@3dd66a, java.io.IOException: Couldn't setup connection for USERNAME to hbase/SERVER
2016-03-03 16:08:14,851 [http-bio-8080-exec-22] WARN  client.HConnectionManager$HConnectionImplementation  - Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:
Thu Mar 03 15:59:02 PST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@db5183, java.io.IOException: Call to SERVERIP:60020 failed on local exception: java.io.EOFException
2016-03-04 00:00:29,875 [http-bio-8080-exec-13] ERROR security.UserGroupInformation  - PriviledgedActionException as:USERNAME (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Attempt to obtain new INITIATE credentials failed! (null))]
2016-03-04 00:00:29,876 [http-bio-8080-exec-13] WARN  security.UserGroupInformation  - Not attempting to re-login since the last re-login was attempted less than 600 seconds before.

我停止了一个服务器，认为可能无法处理更多的连接。但是，我想知道为什么每天都失去联系。我仍在观察我的申请，以找出根本原因。谢谢

hbaseapache-zookeeperkerberos

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com