我用的是Kafka2.12-1.1.0。下面是我的保留配置。我正在使用win os。
log.retention.ms=120000
log.segment.bytes=2800
log.retention.check.interval.ms=30000我看到日志被滚动并生成0000.log到0069.log，然后是*00103.log等等，然后我希望它删除旧文件，但当保留逻辑启动时，我发现错误进程无法访问该文件，因为它正被另一个进程使用，kafka服务器崩溃并且无法从中恢复。当试图删除00000000000000.log时会发生这种情况？你能让我知道的命令，以检查他们是否是活动段或为什么它试图删除这个和崩溃。
如果你想查看pfb日志文件。
[ProducerStateManager partition=myLocalTopic-0] Writing producer snapshot at offset 39
[Log partition=myLocalTopic-0, dir=D:\tmp\kafka-logs] Rolled new log segment at offset 39 in 39 ms.
[ProducerStateManager partition=myLocalTopic-0] Writing producer snapshot at offset 69
[Log partition=myLocalTopic-0, dir=D:\tmp\kafka-logs] Rolled new log segment at offset 69 in 52 ms.
[ProducerStateManager partition=myLocalTopic-0] Writing producer snapshot at offset 91
[Log partition=myLocalTopic-0, dir=D:\tmp\kafka-logs] Rolled new log segment at offset 91 in 18 ms.
[Log partition=myLocalTopic-0, dir=D:\tmp\kafka-logs] Found deletable segments with base offsets [0,69] due to retention time 120000ms breach (kafka.log.Log)
[ProducerStateManager partition=myLocalTopic-0] Writing producer snapshot at offset 103 (kafka.log.ProducerStateManager)
[Log partition=myLocalTopic-0, dir=D:\tmp\kafka-logs] Rolled new log segment at offset 103 in 25 ms. (kafka.log.Log)
[Log partition=myLocalTopic-0, dir=D:\tmp\kafka-logs] Scheduling log segment [baseOffset 0, size 2746] for deletion. (kafka.log.Log)
ERROR Error while deleting segments for myLocalTopic-0 in dir D:\tmp\kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: D:\tmp\kafka-logs\myLocalTopic-0\00000000000000000000.log -> D:\tmp\kafka-logs\myLocalTopic-0\00000000000000000000.log.deleted: The process cannot access the file because it is being used by another process.
............. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:98)
.......................................................
ERROR Uncaught exception in scheduled task 'kafka-log-retention' (kafka.utils.KafkaScheduler)
org.apache.kafka.common.errors.KafkaStorageException: Error while deleting segments for myLocalTopic-0 in dir D:\tmp\kafka-logs
Caused by: java.nio.file.FileSystemException: D:\tmp\kafka-logs\myLocalTopic-0\00000000000000000000.log -> D:\tmp\kafka-logs\myLocalTopic-0\00000000000000000000.log.deleted: The process cannot access the file because it is being used by another process.单击此处查看文件列表图像
1条答案
按热度按时间u59ebvdq1#
apachekafka为我们提供了以下保留策略
基于时间的保留
基于大小的保留
对于基于时间的保留，请尝试以下命令：，
设置保留时间的命令
删除主题级保留时间的命令
对于基于大小的保留，请尝试以下命令来设置保留大小：
要删除，