我们的spark上有一些错误,我们的spark-warn模型无法解决任何问题

mzaanser  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(201)

系统在下面的行中报告错误日志提示,它说我们的配置单元系统中存在一些问题,但是我在配置单元配置文件中查看了很多次,没有发现任何问题:

================================== Traceback (most recent call last):   File "/data/spark/python/pyspark/sql/utils.py", line 63, in deco
     return f(*a,**kw)   File "/data/spark/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line
 319, in get_return_value py4j.protocol.Py4JJavaError: An error
 occurred while calling o22.sessionState. :
 java.lang.IllegalArgumentException: Error while instantiating
 'org.apache.spark.sql.hive.HiveSessionState':
         at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:981)
         at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
         at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:606)
         at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
         at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
         at py4j.Gateway.invoke(Gateway.java:280)
         at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
         at py4j.commands.CallCommand.execute(CallCommand.java:79)
         at py4j.GatewayConnection.run(GatewayConnection.java:214)
         at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.reflect.InvocationTargetException
         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
         at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
         ... 13 more Caused by: java.lang.IllegalArgumentException: Error while instantiating
 'org.apache.spark.sql.hive.HiveExternalCatalog':
         at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:169)
         at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
         at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
         at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
         at scala.Option.getOrElse(Option.scala:121)
         at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
         at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
         at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
         at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
         ... 18 more Caused by: java.lang.NoSuchMethodException: org.apache.spark.sql.hive.HiveExternalCatalog.<init>(org.apache.spark.SparkConf,
 org.apache.hadoop.conf.Configuration)
         at java.lang.Class.getConstructor0(Class.java:2892)
         at java.lang.Class.getDeclaredConstructor(Class.java:2058)
         at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:164)
         ... 26 more

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题