pig没有使用hcatalog定位配置单元表

lstz6jyr  于 2021-06-25  发布在  Pig
关注(0)|答案(1)|浏览(372)

我使用pig访问通过hcatalog创建的表batting\u数据。在执行此操作时，我遇到一个错误，即找不到所提到的表。不过，这个击球数据表在Hive中可用。我也明白，如果没有提到数据库名称，那么就假定为默认值。
error org.apache.pig.tools.grunt.grunt-错误1115:找不到表：default.batting\未找到数据表
我已经设置了hive-site.xml，如下所示。请注意，我使用的不是metastore的远程服务器，而是mysql的本地服务器

<configuration>
<property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
        <description>the URL of the MySQL database</description>
</property>

<property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
</property>

<property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
</property>

<property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>root</value>
</property>

<property>
        <name>hive.hwi.listen.host</name>
        <value>0.0.0.0</value>
</property>
<property>
        <name>hive.hwi.listen.port</name>
        <value>9999</value>
</property>
<property>
        <name>hive.hwi.war.file</name>
        <value>lib/hive-hwi-0.12.0.war</value>
</property>

<property>
        <name>hive.metastore.local</name>
        <value>true</value>
</property>

我已经在我的.bashrc中为pig与hive和hcatalog的集成设置了以下内容。
export pig\u opts=-dhive.metastore.local=true export pig\u classpath=$hcat\u home/share/hcatalog/：$hive\u home/lib/
当清管器启动时，默认情况下将加载下面的语句。
register/home/shiva/hive-0.12.0/hcatalog/share/hcatalog/hcatalog-core-0.12.0.jar；register/home/shiva/hive-0.12.0/lib/hive-exec-0.12.0.jar；register/home/shiva/hive-0.12.0/lib/hive-metastore-0.12.0.jar；
下面是错误消息的完整日志。如有任何帮助，我们将不胜感激。谢谢。

grunt> a = LOAD 'batting_data' USING org.apache.hcatalog.pig.HCatLoader();         
2015-01-01 01:06:33,849 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2015-01-01 01:06:33,865 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize called
2015-01-01 01:06:34,049 [main] INFO  DataNucleus.Persistence - Property datanucleus.cache.level2 unknown - will be ignored
2015-01-01 01:06:34,365 [main] WARN  com.jolbox.bonecp.BoneCPConfig - Max Connections < 1. Setting to 20
2015-01-01 01:06:35,470 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2015-01-01 01:06:35,501 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
2015-01-01 01:06:36,265 [main] WARN  com.jolbox.bonecp.BoneCPConfig - Max Connections < 1. Setting to 20
2015-01-01 01:06:36,506 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore - 0: get_database: NonExistentDatabaseUsedForHealthCheck
2015-01-01 01:06:36,506 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit - ugi=shiva   ip=unknown-ip-addr  cmd=get_database: NonExistentDatabaseUsedForHealthCheck 
2015-01-01 01:06:36,512 [main] ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler - NoSuchObjectException(message:There is no database named nonexistentdatabaseusedforhealthcheck)
    at org.apache.hadoop.hive.metastore.ObjectStore.getMDatabase(ObjectStore.java:431)
    at org.apache.hadoop.hive.metastore.ObjectStore.getDatabase(ObjectStore.java:441)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:124)
    at com.sun.proxy.$Proxy6.getDatabase(Unknown Source)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStore.java:628)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:103)
    at com.sun.proxy.$Proxy7.get_database(Unknown Source)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:810)
    at org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:277)
    at org.apache.hcatalog.common.HiveClientCache.get(HiveClientCache.java:147)
    at org.apache.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:547)
    at org.apache.hcatalog.pig.PigHCatUtil.getHiveMetaClient(PigHCatUtil.java:150)
    at org.apache.hcatalog.pig.PigHCatUtil.getTable(PigHCatUtil.java:186)
    at org.apache.hcatalog.pig.HCatLoader.getSchema(HCatLoader.java:194)
    at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
    at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
    at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:853)
    at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3479)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536)
    at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
    at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
    at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
    at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1648)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1621)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:541)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

2015-01-01 01:06:36,514 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore - 0: get_table : db=default tbl=batting_data
2015-01-01 01:06:36,514 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore.audit - ugi=shiva   ip=unknown-ip-addr  cmd=get_table : db=default tbl=batting_data 
2015-01-01 01:06:36,516 [main] INFO  DataNucleus.Datastore - The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2015-01-01 01:06:36,516 [main] INFO  DataNucleus.Datastore - The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2015-01-01 01:06:36,795 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1115: Table not found : default.batting_data table not found
Details at logfile: /home/shiva/pig_1420054544179.log
Hiveapache-pighcatalog

1条答案

按热度按时间
qco9c6ql

qco9c6ql1#

好了，开始。。我修好了。。
我没有提到pig的opts与hive-thrift服务器的正确地址，因为pig无法连接到hive-metastore，并且找不到相应的表。改成 PIG_OPTS=-Dhive.metastore.uris=thrift://localhost:10000 使用启动hiveserver服务
$bin/hive—服务hiveserver
以上解决了这个问题，现在我可以连接Pig到Hive。谢谢

赞(0）分享  回复(0）举报  2021-06-25
我来回答

相关问题

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com