查询连接到hbase的配置单元外部表时,spark挂起

shstlldc  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(203)

我们已经创建了一个连接到hbase的配置单元外部表,但是当我们尝试在sparkshell中访问该表时,只需执行以下简单查询,
scala>sqlcontext.sql(“select*from[hive external table]).count()
日志的后半部分如下所示:

15/10/13 17:13:08 INFO ClientCnxn: Opening socket connection to server BData-h1/10.10.10.82:2181. Will not attempt to authenticate using SASL (unknown error)
15/10/13 17:13:08 INFO ClientCnxn: Socket connection established, initiating session, client: /10.10.10.82:34108, server: BData-h1/10.10.10.82:2181
15/10/13 17:13:08 INFO ClientCnxn: Session establishment complete on server BData-h1/10.10.10.82:2181, sessionid = 0x1505f6805c70281, negotiated timeout = 60000
15/10/13 17:13:09 INFO RegionSizeCalculator: Calculating region sizes for table "analytics_demo".
15/10/13 17:13:24 INFO SparkContext: Starting job: collect at SparkPlan.scala:83
15/10/13 17:13:24 INFO DAGScheduler: Got job 0 (collect at SparkPlan.scala:83) with 1 output partitions (allowLocal=false)
15/10/13 17:13:24 INFO DAGScheduler: Final stage: Stage 0(collect at SparkPlan.scala:83)
15/10/13 17:13:24 INFO DAGScheduler: Parents of final stage: List()
15/10/13 17:13:24 INFO DAGScheduler: Missing parents: List()
15/10/13 17:13:24 INFO DAGScheduler: Submitting Stage 0 (MapPartitionsRDD[7] at map at SparkPlan.scala:83), which has no missing parents
15/10/13 17:13:24 INFO MemoryStore: ensureFreeSpace(16512) called with curMem=601741, maxMem=278302556
15/10/13 17:13:24 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 16.1 KB, free 264.8 MB)
15/10/13 17:13:24 INFO MemoryStore: ensureFreeSpace(8676) called with curMem=618253, maxMem=278302556
15/10/13 17:13:24 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 8.5 KB, free 264.8 MB)
15/10/13 17:13:24 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on BData-h1:35111 (size: 8.5 KB, free: 265.4 MB)
15/10/13 17:13:24 INFO BlockManagerMaster: Updated info of block broadcast_1_piece0
15/10/13 17:13:24 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:839
15/10/13 17:13:24 INFO DAGScheduler: Submitting 1 missing tasks from Stage 0 (MapPartitionsRDD[7] at map at SparkPlan.scala:83)
15/10/13 17:13:24 INFO YarnScheduler: Adding task set 0.0 with 1 tasks
15/10/13 17:13:25 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, BData-h2, RACK_LOCAL, 1424 bytes)
15/10/13 17:13:25 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on BData-h2:37159 (size: 8.5 KB, free: 530.3 MB)
15/10/13 17:13:27 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on BData-h2:37159 (size: 42.8 KB, free: 530.2 MB)

系统就挂在那里。
如果我们查询一个配置单元管理的表,它将很好地返回结果。
我们尝试了很多事情,但是没有运气。有人能为我们解释一下这个问题吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题