抛出错误

gxwragnw  于 2021-05-29  发布在  Hadoop
关注(0)|答案(3)|浏览(345)

尝试在storm trident中将xml转换为json我的代码在storm 0.9.1版本中运行良好，现在我们升级到了storm 0.10.0的最新版本，我们得到了以下错误如果您对原因有任何想法，请让我知道？

11752 [Thread-56-b-1] ERROR b.s.util - Async loop died!
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.JavaType.isReferenceType()Z
    at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findDefaultDeserializer(BasicDeserializerFactory.java:1522) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.findStdDeserializer(BeanDeserializerFactory.java:167) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:132) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:403) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:352) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:461) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3804) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3698) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2714) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.tda.trident.bolt.HdfsStateConvertor$HdfsFileOptions.execute(HdfsStateConvertor.java:278) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.tda.trident.bolt.HdfsStateConvertor.updateState(HdfsStateConvertor.java:462) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.tda.trident.bolt.HdfsUpdater.updateState(HdfsUpdater.java:15) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at com.tda.trident.bolt.HdfsUpdater.updateState(HdfsUpdater.java:9) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?]
    at storm.trident.planner.processor.PartitionPersistProcessor.finishBatch(PartitionPersistProcessor.java:98) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at storm.trident.planner.SubtopologyBolt.finishBatch(SubtopologyBolt.java:152) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at storm.trident.topology.TridentBoltExecutor.finishBatch(TridentBoltExecutor.java:253) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at storm.trident.topology.TridentBoltExecutor.checkFinish(TridentBoltExecutor.java:286) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at storm.trident.topology.TridentBoltExecutor.execute(TridentBoltExecutor.java:360) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.daemon.executor$fn__6214$tuple_action_fn__6216.invoke(executor.clj:670) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.daemon.executor$mk_task_receiver$fn__6137.invoke(executor.clj:426) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.disruptor$clojure_handler$reify__5713.onEvent(disruptor.clj:58) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.daemon.executor$fn__6214$fn__6227$fn__6278.invoke(executor.clj:808) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at backtype.storm.util$async_loop$fn__543.invoke(util.clj:475) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
    at java.lang.Thread.run(Thread.java:744) [?:1.7.0_45]

在构建pom时，我的maven shade输出如下所示。

[INFO] --- maven-shade-plugin:1.4:shade (default) @ StormTridentXmlNew ---
[INFO] Including org.apache.hadoop:hadoop-auth:jar:2.7.1.2.3.2.0-2950 in the shaded jar.
[INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.storm:storm-hdfs:jar:0.10.0.2.3.2.0-2950 in the shaded jar.
[INFO] Including org.yaml:snakeyaml:jar:1.11 in the shaded jar.
[INFO] Including com.hmsonline:storm-jms:jar:0.2.2 in the shaded jar.
[INFO] Including org.apache.geronimo.specs:geronimo-jms_1.1_spec:jar:1.1.1 in the shaded jar.
[INFO] Including com.tibco.ems:tibjms:jar:5.1 in the shaded jar.
[INFO] Including org.springframework:spring-beans:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including org.springframework:spring-core:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including commons-logging:commons-logging:jar:1.2 in the shaded jar.
[INFO] Including org.springframework:spring-context:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including org.springframework:spring-aop:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
[INFO] Including org.springframework:spring-expression:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including org.springframework:spring-jms:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including org.springframework:spring-messaging:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including org.springframework:spring-tx:jar:4.1.3.RELEASE in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-core:jar:2.6.5 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-databind:jar:2.6.5 in the shaded jar.
[INFO] Including com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.6.5 in the shaded jar.
[INFO] Including com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.6.5 in the shaded jar.
[INFO] Including org.codehaus.woodstox:stax2-api:jar:3.1.4 in the shaded jar.
[INFO] Including org.codehaus.woodstox:woodstox-core-asl:jar:4.4.1 in the shaded jar.
[INFO] Including javax.xml.stream:stax-api:jar:1.0-2 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-client:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.1 in the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-yarn-client:jar:2.7.1 from the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.1 in the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-yarn-common:jar:2.7.1 from the shaded jar.
[INFO] Including javax.xml.bind:jaxb-api:jar:2.2.2 in the shaded jar.
[INFO] Including javax.activation:activation:jar:1.1 in the shaded jar.
[INFO] Including com.sun.jersey:jersey-client:jar:1.9 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-annotations:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-common:jar:2.7.1 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:11.0.2 in the shaded jar.
[INFO] Including commons-cli:commons-cli:jar:1.2 in the shaded jar.
[INFO] Including org.apache.commons:commons-math3:jar:3.1.1 in the shaded jar.
[INFO] Including xmlenc:xmlenc:jar:0.52 in the shaded jar.
[INFO] Including commons-httpclient:commons-httpclient:jar:3.1 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.4 in the shaded jar.
[INFO] Including commons-io:commons-io:jar:2.4 in the shaded jar.
[INFO] Including commons-net:commons-net:jar:3.1 in the shaded jar.
[INFO] Including commons-collections:commons-collections:jar:3.2.1 in the shaded jar.
[INFO] Including javax.servlet:servlet-api:jar:2.5 in the shaded jar.
[INFO] Including org.mortbay.jetty:jetty:jar:6.1.26 in the shaded jar.
[INFO] Including org.mortbay.jetty:jetty-util:jar:6.1.26 in the shaded jar.
[INFO] Including javax.servlet.jsp:jsp-api:jar:2.1 in the shaded jar.
[INFO] Including com.sun.jersey:jersey-core:jar:1.9 in the shaded jar.
[INFO] Including com.sun.jersey:jersey-json:jar:1.9 in the shaded jar.
[INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar.
[INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar.
[INFO] Including com.sun.jersey:jersey-server:jar:1.9 in the shaded jar.
[INFO] Including asm:asm:jar:3.1 in the shaded jar.
[INFO] Including log4j:log4j:jar:1.2.17 in the shaded jar.
[INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar.
[INFO] Including org.apache.httpcomponents:httpclient:jar:4.1.2 in the shaded jar.
[INFO] Including org.apache.httpcomponents:httpcore:jar:4.1.2 in the shaded jar.
[INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including commons-configuration:commons-configuration:jar:1.6 in the shaded jar.
[INFO] Including commons-digester:commons-digester:jar:1.8 in the shaded jar.
[INFO] Excluding commons-beanutils:commons-beanutils:jar:1.7.0 from the shaded jar.
[INFO] Excluding commons-beanutils:commons-beanutils-core:jar:1.8.0 from the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.10 from the shaded jar.
[INFO] Excluding org.slf4j:slf4j-log4j12:jar:1.7.10 from the shaded jar.
[INFO] Including org.apache.avro:avro:jar:1.7.4 in the shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.3 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.0.4.1 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.2.4 in the shaded jar.
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar.
[INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar.
[INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.6 in the shaded jar.
[INFO] Including io.netty:netty:jar:3.7.0.Final in the shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded jar.
[INFO] Including org.tukaani:xz:jar:1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.1 in the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar.
[INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar.
[INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
hadoopJSONxmlJacksonapache-storm

3条答案

按热度按时间
vom3gejh

vom3gejh1#

storm的最新版本在lib文件夹中有jackson-core-2.3.1.jar，我用maven创建的胖jar有另一个版本的jackson依赖项。这就是为什么当我在storm集群中提交jar时，它会在运行时抛出这些没有这样的方法错误。。
我改变了我的pom使用相同版本的storm，一切正常。

赞(0）分享  回复(0）举报  2021-05-30
qvk1mo1f

qvk1mo1f2#

看起来您有许多版本的jacksonjar，并且您的类路径在运行时引用了2.6之前的jacksonjar。
从版本2.6开始，类resolvedtype引入了isreferencetype（）。请参考以下链接：
http://googleweblight.com/?lite_url=http：//fasterxml.github.io/jackson core/javadoc/2.6/com/fasterxml/jackson/core/type/resolvedtype.html&ei=ochb8ipc&lc=en in&geid=10&s=1&m=71&ts=1454581842&sig=all1aj7i2eau3aokmou6fkbtfttuku02bg#a-isreferencetype（）
请确保在运行时引用正确的版本或更高版本。如果不需要，您可以移除/删除旧的jackson jar。
希望这有帮助。

赞(0）分享  回复(0）举报  2021-05-30
cld4siwp

cld4siwp3#

这个问题似乎来自classpath中jackson类的多个版本。您可以删除不需要的版本。在写这篇文章的时候。jackson的当前版本是isis 2.7.1。

赞(0）分享  回复(0）举报  2021-05-29
我来回答

相关问题

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com