kafka streams:“更新的集群元数据版本”运行了很长时间

nxowjjhe  于 2021-06-07  发布在  Kafka
关注(0)|答案(0)|浏览(310)

在过去的几个小时里,我的工作Kafka流试图旋转起来,但被占用 Updated cluster metadata version 对于一个非常大的主题:

2018-01-03T13:30:56.871 DEBUG [org.apache.kafka.clients.consumer.KafkaConsumer] - Starting the Kafka consumer
2018-01-03T13:30:56.882 DEBUG [org.apache.kafka.clients.Metadata] - Updated cluster metadata version 1 to Cluster(id = null, nodes = [node2:9092 (id: -2 rack: null), 
  node3:9092 (id: -3 rack: null), node1:9092 (id: -1 rack: null)], partitions = [])
...
2018-01-04T10:10:27.945 DEBUG [org.apache.kafka.clients.NetworkClient] - Sending metadata request (type=MetadataRequest, topics=<ALL>) to node node3:9092 (id: 3 rack: null)
2018-01-04T10:10:27.952 DEBUG [org.apache.kafka.clients.Metadata] - Updated cluster metadata version 704157 to Cluster(id = DmwzpJYRTEGrm8MfjcpXLw, nodes = [node3:9092 (id: 3 rack: null), 
  node1:9092 (id: 1 rack: null), node2:9092 (id: 2 rack: null)], partitions = [Partition(topic = raw-messages, partition = 9, leader = 3, replicas = [3], isr = [3]), 
  Partition(topic = raw-messages, partition = 8, leader = none, replicas = [2], isr = []), 
  Partition(topic = raw-messages, partition = 7, leader = 1, replicas = [1], isr = [1]), 
  Partition(topic = raw-messages, partition = 6, leader = 3, replicas = [3], isr = [3]), 
  Partition(topic = raw-messages, partition = 5, leader = none, replicas = [2], isr = []), 
  Partition(topic = raw-messages, partition = 4, leader = 1, replicas = [1], isr = [1]), 
  Partition(topic = raw-messages, partition = 3, leader = 3, replicas = [3], isr = [3]), 
  Partition(topic = raw-messages, partition = 2, leader = none, replicas = [2], isr = []), 
  Partition(topic = raw-messages, partition = 1, leader = 1, replicas = [1], isr = [1]), 
  Partition(topic = raw-messages, partition = 0, leader = 3, replicas = [3], isr = [3])])

这是一个新的kafka streams作业,具有新的应用程序id,应该从头开始(offset=0)。
有几个事件可能导致这种行为:
我们从融合平台3.2.0迁移到融合平台4.0.0
由于数据中心断电,Kafka群集严重崩溃
我的问题是,我应该让它运行吗?目前,该工作在 metadata version 704157 ,我可以找出或估计版本总数吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题