apache kafka分区,java

nhhxz33t  于 2021-06-08  发布在  Kafka
关注(0)|答案(1)|浏览(260)

我对ApacheKafka非常陌生,我正在尝试执行kafka分区,同时将字符串发送给kafka,
做的和这里完全一样

public class Producer {
    public static void main(String[] argv)throws Exception {

           String topicName = "test";

           //Configure the Producer
           Properties configProperties = new Properties();
           configProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"192.168.4.226:9092");
           configProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.ByteArraySerializer");
           configProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");
           configProperties.put(ProducerConfig.PARTITIONER_CLASS_CONFIG,CountryPartitioner.class.getCanonicalName());
           configProperties.put("partitions.0","USA");
           configProperties.put("partitions.1","India"); 

           org.apache.kafka.clients.producer.Producer producer = new KafkaProducer(configProperties);
               ProducerRecord<String, String> rec = new ProducerRecord<String, String>(topicName,"message111");
               producer.send(rec, new Callback() {
                   public void onCompletion(RecordMetadata metadata, Exception exception) {
                       System.out.println("Message sent to topic ->" + metadata.topic() +" stored at offset->" + metadata.offset());
                   }
               });
           producer.close();
       }

}

现在我遇到了这样一个例外

2018-07-03 17:52:12 ERROR RecordBatch:102 - Error executing user-provided callback on message for topic-partition test-2:
java.lang.NullPointerException
    at com.spnotes.kafka.partition.Producer$1.onCompletion(Producer.java:29)
    at org.apache.kafka.clients.producer.internals.RecordBatch.done(RecordBatch.java:99)
    at org.apache.kafka.clients.producer.internals.RecordBatch.maybeExpire(RecordBatch.java:136)
    at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortExpiredBatches(RecordAccumulator.java:220)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:192)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:141)
    at java.lang.Thread.run(Thread.java:748)

我做错了什么?提前谢谢。

0aydgbwb

0aydgbwb1#

编辑3:唯一的问题是您必须创建多个分区,请参阅以下步骤。首先,我们为每个代理创建一个配置文件(在windows上使用copy命令)

> cp config/server.properties config/server-1.properties
> cp config/server.properties config/server-2.properties

现在编辑这些新文件并设置以下属性:

config/server-1.properties:
    broker.id=1
    listeners=PLAINTEXT://:9093
    log.dir=/tmp/kafka-logs-1

config/server-2.properties:
    broker.id=2
    listeners=PLAINTEXT://:9094
    log.dir=/tmp/kafka-logs-2

我们已经启动了zookeeper和单个节点,所以我们只需要启动两个新节点:

> bin/kafka-server-start.sh config/server-1.properties &
> bin/kafka-server-start.sh config/server-2.properties &

现在创建一个新主题,其中复制因子为3,并带有3个分区,这将解决问题:

> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic my-replicated-topic

相关问题