csv Java ZipOutputStream创建部分重复内容的损坏文件

mf98qq94  于 5个月前  发布在  Java
关注(0)|答案(1)|浏览(53)

Sping Boot 3项目(除了其他事情)将不得不保存一些CSV文件到磁盘,然后还创建一个ZIP从这些文件到另一个文件夹作为备份。CSV文件生成正确。我使用JacksonCSVMap器为每个文件单独生成CSV内容(3个基类),然后ZipOutputStream将所有3个文件放入一个zip中。在磁盘上检查时,得到的CSV文件是正确的,但是zip中的同一文件不正确。从原始文件中提取正确的CSV内容示例:

"ID";"DATE";"REC_STATUS";"ORIG_ID";"PARAM_NAME";"PARAM_VALUE"
"109";"2023-11-27";"0";"116";"transfer1 param";"transfer1 value"
"110";"2023-11-27";"0";"116";"transfer2";"transfer2 value2"
"111";"2023-11-27";"0";"117";"transfer1 param";"transfer1 value"
"112";"2023-11-27";"0";"117";"transfer2";"transfer2 value2"
"113";"2023-11-27";"0";"118";"transfer1 param";"transfer1 value"
"114";"2023-11-27";"0";"118";"transfer2";"transfer2 value2"
"115";"2023-11-27";"0";"119";"transfer1 param";"transfer1 value"
"116";"2023-11-27";"0";"119";"transfer2";"transfer2 value2"
"117";"2023-11-27";"0";"120";"param name1";"param1 value1"
"118";"2023-11-27";"0";"120";"name2";"value2"
"119";"2023-11-27";"0";"121";"param name1";"param1 value1"
"120";"2023-11-27";"0";"121";"name2";"value2"
"121";"2023-11-27";"0";"122";"param name1";"param1 value1"
"122";"2023-11-27";"0";"122";"name2";"value2"
"123";"2023-11-27";"0";"123";"param name1";"param1 value1"
"124";"2023-11-27";"0";"123";"name2";"value2"
"125";"2023-11-27";"0";"124";"param name1";"param1 value1"
"126";"2023-11-27";"0";"124";"name2";"value2"

字符串
从zip中取出的相同文件包含损坏的数据:

"ID";"DATE";"REC_STATUS";"ORIG_ID";"PARAM_NAME";"PARAM_VALUE"
"109";"2023-11-27";"0";"116";"transfer1 param";"transfer1 value"
"110";"2023-11-27";"0";"116";"transfer2";"transfer2 value2"
"111";"2023-11-27";"0";"117";"transfer1 param";"transfer1 value"
"112";"2023-11-27";"0";"117";"transfer2";"transfer2 value2"
"113";"2023-11-27";"0";"118";"transfer1 param";"transfer1 value"
"114";"2023-11-27";"0";"118";"transfer2";"transfer2 value2"
"115";"2023-11-27";"0";"119";"transfer1 param";"transfer1 value"
"116";"2023-11-27";"0";"119";"transfer2";"transfer2 value2"
"117";"2023-11-27";"0";"120";"param name1";"param1 value1"
"118";"2023-11-27";"0";"120";"name2";"value2"
"119";"2023-11-27";"0";"121";"param name1";"param1 value1"
"120";"2023-11-27";"0";"121";"name2";"value2"
"121";"2023-11-27";"0";"122";"param name1";"param1 value1"
"122";"2023-11-27";"0";"122";"name2";"value2"
"123";"2023-11-27";"0";"123";"param name1";"param1 value1"
"124";"2023-11-27";"0";"123";"name2";"value2"
"125";"2023-11-27";"0";"124";"param name1";"param1 value1"
"126";"2023-11-27";"0";"124";"name2";"value2"
109";"2023-11-27";"0";"116";"transfer1 param";"transfer1 value"
"110";"2023-11-27";"0";"116";"transfer2";"transfer2 value2"
"111";"2023-11-27";"0";"117";"transfer1 param";"transfer1 value"
"112";"2023-11-27";"0";"117";"transfer2";"transfer2 value2"
"113";"2023-11-27";"0";"118";"transfer1 param";"transfer1 value"
"114";"2023-11-27";"0";"118";"transfer2";"transfer2 value2"
"115";"2023-11-27";"0";"119";"transfer1 param";"transfer1 value"
"116";"2023-11-27";"0";"119";"transfer2";"transfer2 value2"
"117";"2023-11-27";"0";"120";"param name1";"param1 value1"
"118";"2023-11-27";"0";"120";"name2";"value2"
"119";"2023-11-27";"0";"121";"param name1";"param1 value1"
"120";"2023-11-27";"0";"121";"name2";"value2"
"121";"2023-11-27";"0";"122";"param name1";"param1 value1"
"122";"2023-11-27";"0";"122";"name2";"value2"
"123";"2023-11-27";"0";"123";"param name1";"param1 value1"
"124";"2023-11-27";"0";"123";"name2";"value2"
"


请注意CSV文件似乎在中间重复,但在行中没有前导“,并且在最后一行中也有一个unneeded“。
CSV生成:

private String collectValuesToCSV(List<? extends Object> values, CsvMapper csvMapper, CsvSchema csvSchema) {
    String result;
    
    try (StringWriter strW = new StringWriter();
            SequenceWriter seqW = csvMapper.writer(csvSchema).writeValues(strW)) {
        for (Object value : values) {
            seqW.write(value);
        }
        
        seqW.flush();
        strW.flush();
    
        result = strW.toString();
    } catch (IOException e) {
        log.fatal(String.format("%s - Unable to generate CSV", LOG_PREFIX), e);
        throw e;
    }
    
    return result;
}

// usage in other parts of the code
CsvMapper csvMapper = CsvMapper.builder()
        .enable(CsvGenerator.Feature.ALWAYS_QUOTE_STRINGS)
        .enable(CsvGenerator.Feature.ALWAYS_QUOTE_EMPTY_STRINGS)
        .build();

CsvSchema csvSchema = csvMapper
        .schemaFor(MyDTO.class)
        .withColumnSeparator(';')
        .withHeader();

List<MyDTO> objects = generateObjects();

String myCsv = collectValuesToCSV(objects, csvMapper, csvSchema);

byte[] contents = myCsv.getBytes(StandardCharsets.UTF_8);

// optionally the contents may be encrypted, so a byte array is written to disk,
// but the same problem occurs even without encryption, simply writing this byte[] to disk

Path filePath = Path.of(config.baseFolder, "output_" + dateStr + ".csv");
Files.write(filePath, contents);


创建ZIP文件:

private void zipCsvFiles(String dateStr, Path path1, Path path2, Path path3) throws IOException {
    String zipFileName = Path.of(config.zipFolder, "output_" + dateStr + ".zip").toString();
    
    try (FileOutputStream fos = new FileOutputStream(zipFileName);
            ZipOutputStream zipOut = new ZipOutputStream(fos)) {
        addZipEntry(zipOut, path1);
        addZipEntry(zipOut, path2);
        addZipEntry(zipOut, path3);
    }
}

private void addZipEntry(ZipOutputStream zipOut, Path path) throws IOException {
    try (FileInputStream fis = new FileInputStream(path.toFile()) {
        ZipEntry entry = new ZipEntry(path.getFileName().toString());
        zipOut.putNextEntry(entry);

        byte[] bytes = new byte[1024];

        while ((fis.read(bytes)) >= 0) {
            zipOut.write(bytes);
        }
        
        zipOut.closeEntry();
    }
}


结果压缩看起来不错,它显示了3个文件,文件名正确。但是当打开其中任何一个文件时,内容都被破坏了......我研究了关于这个主题的其他问题,并尝试了一些想法(手动调用流上的flush和finish),但据我所知,这些都是多余的,因为close()方法会根据需要调用它们,而close()方法总是在我处理try-with-resources中的所有事情时被调用。所以我不确定是什么导致了这个问题。

kzipqqlq

kzipqqlq1#

问题出现在addZipEntry中的以下行中:

byte[] bytes = new byte[1024];

    while ((fis.read(bytes)) >= 0) {
        zipOut.write(bytes);
    }

字符串
方法read返回读取的字节数,但您不使用它,而是始终将整个数组发送到zipOut
将其更改为以下内容:

byte[] bytes = new byte[1024];
    int read = fis.read(bytes);
            
    while (read >= 0) {
        zipOut.write(bytes, 0, read);
        read = fis.read(bytes);
    }

相关问题