mesos下的齐柏林飞艇-初始作业未接受任何资源

mbskvtky  于 2021-06-15  发布在  Cassandra
关注(0)|答案(0)|浏览(250)

尝试在mesos集群上使用zeppelin运行查询,使用cassandra作为数据源。
我可以在dev环境中运行它,但是当使用相同的配置和包进行prod时,它就不起作用了。
我使用下面的代码来创建表结构,这个代码传递:

import org.apache.spark.sql.SparkSession

val sparkSession = SparkSession.builder().getOrCreate()

val state = sparkSession.read.format("org.apache.spark.sql.cassandra").options(Map("keyspace" -> "someKeySpace", "table" -> "someTable")).load() state.createOrReplaceTempView("state")

在运行查询的下一段中:

%sql

select 1 from state

任务只是挂起,以0%运行,什么也没有发生,在尝试调试时,我在解释器日志中得到以下信息,我提供了足够的cpu/mem资源,并且所有nw通信都启用了master/slave

DEBUG [2018-10-29 10:25:43,915] ({dispatcher-event-loop-15} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:44,915] ({dispatcher-event-loop-0} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:45,915] ({dispatcher-event-loop-2} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:46,553] ({Thread-436} Logging.scala[logDebug]:58) - Received 2 resource offers.
DEBUG [2018-10-29 10:25:46,553] ({Thread-436} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22973 with attributes: Map() mem: 32218.0 cpu: 2.9 port: List((1025,2180), (
2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,9000), (9002,9041), (9043,9159), (9161,9599), (9601,27059), (27061,32000))
DEBUG [2018-10-29 10:25:46,553] ({Thread-436} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22974 with attributes: Map() mem: 50650.0 cpu: 10.9 port: List((1025,2180),
(2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,9000), (9002,9041), (9043,9159), (9161,9599), (9601,14175), (14177,32000))
DEBUG [2018-10-29 10:25:46,915] ({dispatcher-event-loop-1} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:47,915] ({dispatcher-event-loop-4} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:48,232] ({Thread-437} Logging.scala[logDebug]:58) - Received 1 resource offers.
DEBUG [2018-10-29 10:25:48,232] ({Thread-437} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22977 with attributes: Map() mem: 37338.0 cpu: 4.9 port: List((1025,2180), (
2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,8999), (9002,9041), (9043,9159), (9161,9599), (9601,20592), (20597,32000))
DEBUG [2018-10-29 10:25:48,915] ({dispatcher-event-loop-3} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:49,915] ({dispatcher-event-loop-6} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:50,915] ({dispatcher-event-loop-5} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0 
DEBUG [2018-10-29 10:25:51,915] ({dispatcher-event-loop-7} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:52,241] ({Thread-438} Logging.scala[logDebug]:58) - Received 2 resource offers.
DEBUG [2018-10-29 10:25:52,241] ({Thread-438} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22979 with attributes: Map() mem: 32218.0 cpu: 2.9 port: List((1025,2180), (
2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,9000), (9002,9041), (9043,9159), (9161,9599), (9601,27059), (27061,32000))
DEBUG [2018-10-29 10:25:52,241] ({Thread-438} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22980 with attributes: Map() mem: 50650.0 cpu: 10.9 port: List((1025,2180),
(2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,9000), (9002,9041), (9043,9159), (9161,9599), (9601,14175), (14177,32000))
DEBUG [2018-10-29 10:25:52,915] ({dispatcher-event-loop-8} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:53,243] ({Thread-439} Logging.scala[logDebug]:58) - Received 1 resource offers.
DEBUG [2018-10-29 10:25:53,243] ({Thread-439} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22982 with attributes: Map() mem: 37338.0 cpu: 4.9 port: List((1025,2180), (
2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,8999), (9002,9041), (9043,9159), (9161,9599), (9601,20592), (20597,32000))
DEBUG [2018-10-29 10:25:53,915] ({dispatcher-event-loop-9} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:54,915] ({dispatcher-event-loop-10} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:55,915] ({dispatcher-event-loop-11} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:56,915] ({dispatcher-event-loop-12} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
DEBUG [2018-10-29 10:25:57,251] ({Thread-440} Logging.scala[logDebug]:58) - Received 2 resource offers.
DEBUG [2018-10-29 10:25:57,252] ({Thread-440} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22986 with attributes: Map() mem: 32218.0 cpu: 2.9 port: List((1025,2180), (
2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,9000), (9002,9041), (9043,9159), (9161,9599), (9601,27059), (27061,32000))
DEBUG [2018-10-29 10:25:57,252] ({Thread-440} Logging.scala[logDebug]:58) - Declining offer: 7ed1dd50-6efe-422d-913c-4835a3db28f4-O22987 with attributes: Map() mem: 50650.0 cpu: 10.9 port: List((1025,2180),
(2182,3887), (3889,5049), (5052,6999), (7002,7198), (7200,8079), (8082,8180), (8182,9000), (9002,9041), (9043,9159), (9161,9599), (9601,14175), (14177,32000))
DEBUG [2018-10-29 10:25:57,915] ({dispatcher-event-loop-13} Logging.scala[logDebug]:58) - parentName: default, name: TaskSet_0.0, runningTasks: 0
WARN [2018-10-29 10:25:58,182] ({Timer-0} Logging.scala[logWarning]:66) - Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resource

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题