运行结构化流示例“warn networkclient:bootstrap broker ip-10-28-3-35.ec2”时出错internal:2181 disconnected"

gev0vcfq  于 2021-06-08  发布在  Kafka
关注(0)|答案(1)|浏览(460)

跑步时 structured_kafka_wordcount.py 示例我得到以下错误:

WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected

环境:hdp-2.6.5.0
我能够阅读Kafka的一个主题的内容,如 DStream 例子
运行这些作业的命令

bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1 examples/src/main/python/sql/streaming/structured_kafka_wordcount.py ip-10-28-3-35.ec2.internal:2181 subscribe fifa2
[root@centos spark2]# bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1 examples/src/main/python/sql/streaming/structured_kafka_wordcount.py ip-10-28-3-35.ec2.internal:2181 subscribe fifa2
Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
:: loading settings :: url = jar:file:/usr/hdp/2.6.5.0-292/spark2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
org.apache.spark#spark-sql-kafka-0-10_2.11 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
        confs: [default]
        found org.apache.spark#spark-sql-kafka-0-10_2.11;2.3.1 in central
        found org.apache.kafka#kafka-clients;0.10.0.1 in central
        found net.jpountz.lz4#lz4;1.3.0 in central
        found org.xerial.snappy#snappy-java;1.1.2.6 in central
        found org.slf4j#slf4j-api;1.7.16 in central
        found org.spark-project.spark#unused;1.0.0 in central
:: resolution report :: resolve 642ms :: artifacts dl 15ms
        :: modules in use:
        net.jpountz.lz4#lz4;1.3.0 from central in [default]
        org.apache.kafka#kafka-clients;0.10.0.1 from central in [default]
        org.apache.spark#spark-sql-kafka-0-10_2.11;2.3.1 from central in [default]
        org.slf4j#slf4j-api;1.7.16 from central in [default]
        org.spark-project.spark#unused;1.0.0 from central in [default]
        org.xerial.snappy#snappy-java;1.1.2.6 from central in [default]
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   6   |   0   |   0   |   0   ||   6   |   0   |
        ---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent
        confs: [default]
        0 artifacts copied, 6 already retrieved (0kB/17ms)
18/06/22 06:57:26 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
18/06/22 06:57:26 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
18/06/22 06:57:35 WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected
18/06/22 06:57:35 WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected
18/06/22 06:57:35 WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected
18/06/22 06:57:35 WARN NetworkClient: Bootstrap broker ip-10-28-3-35.ec2.internal:2181 disconnected
^Z
[11]+  Stopped                 bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.1 examples/src/main/python/sql/streaming/structured_kafka_wordcount.py ip-10-28-3-35.ec2.internal:2181 subscribe fifa2
ux6nzvsh

ux6nzvsh1#

你需要提供正确的Kafka服务器,而不是Zookeeper

相关问题