如何读取用7z压缩的文本文件?

lyr7nygr  于 2021-08-20  发布在  Java
关注(0)|答案(3)|浏览(352)

我想从一个7z压缩的csv(文本)文件中逐行读取(在Python2.7中)。我不想解压缩整个(大)文件,而是流式传输行。
我试过了 pylzma.decompressobj() 没有成功。我得到一个数据错误。请注意,此代码尚未逐行读取:

input_filename = r"testing.csv.7z"
with open(input_filename, 'rb') as infile:
    obj = pylzma.decompressobj()
    o = open('decompressed.raw', 'wb')
    obj = pylzma.decompressobj()
    while True:
        tmp = infile.read(1)
        if not tmp: break
        o.write(obj.decompress(tmp))
    o.close()

输出:

o.write(obj.decompress(tmp))
ValueError: data error during decompression
4ktjp1zp

4ktjp1zp1#

这将允许您迭代这些行。它部分来自我在回答另一个问题时发现的一些代码。
此时此刻( pylzma-0.5.0 ) py7zlib 模块未实现允许将存档成员作为字节或字符流读取的api—它的 ArchiveFile 类只提供一个 read() 函数,该函数一次解压缩并返回成员中未压缩的数据。有鉴于此,可以做的最好的事情就是通过python生成器以迭代方式返回字节或行,并将其用作缓冲区。
下面是后者,但如果问题是归档成员文件本身太大,则可能没有帮助。
下面的代码应该可以在Python3.x和2.7中使用。

import io
import os
import py7zlib

class SevenZFileError(py7zlib.ArchiveError):
    pass

class SevenZFile(object):
    @classmethod
    def is_7zfile(cls, filepath):
        """ Determine if filepath points to a valid 7z archive. """
        is7z = False
        fp = None
        try:
            fp = open(filepath, 'rb')
            archive = py7zlib.Archive7z(fp)
            _ = len(archive.getnames())
            is7z = True
        finally:
            if fp: fp.close()
        return is7z

    def __init__(self, filepath):
        fp = open(filepath, 'rb')
        self.filepath = filepath
        self.archive = py7zlib.Archive7z(fp)

    def __contains__(self, name):
        return name in self.archive.getnames()

    def readlines(self, name, newline=''):
        r""" Iterator of lines from named archive member.

        `newline` controls how line endings are handled.

        It can be None, '', '\n', '\r', and '\r\n' and works the same way as it does
        in StringIO. Note however that the default value is different and is to enable
        universal newlines mode, but line endings are returned untranslated.
        """
        archivefile = self.archive.getmember(name)
        if not archivefile:
            raise SevenZFileError('archive member %r not found in %r' %
                                  (name, self.filepath))

        # Decompress entire member and return its contents iteratively.
        data = archivefile.read().decode()
        for line in io.StringIO(data, newline=newline):
            yield line

if __name__ == '__main__':

    import csv

    if SevenZFile.is_7zfile('testing.csv.7z'):
        sevenZfile = SevenZFile('testing.csv.7z')

        if 'testing.csv' not in sevenZfile:
            print('testing.csv is not a member of testing.csv.7z')
        else:
            reader = csv.reader(sevenZfile.readlines('testing.csv'))
            for row in reader:
                print(', '.join(row))
c7rzv4ha

c7rzv4ha2#

如果您使用的是Python3.3+,则可以使用 lzma 在该版本中添加到标准库的模块。
见: lzma 例子

ppcbkaq5

ppcbkaq53#

如果您可以使用python 3,那么有一个有用的库py7zr,它支持部分7zip解压缩,如下所示:

import py7zr
import re
filter_pattern = re.compile(r'<your/target/file_and_directories/regex/expression>')
with SevenZipFile('archive.7z', 'r') as archive:
    allfiles = archive.getnames()
    selective_files = [f if filter_pattern.match(f) for f in allfiles]
    archive.extract(targets=selective_files)

相关问题