Search before asking
- I had searched in the issues and found no similar issues.
Description
For more details see https://cwiki.apache.org/confluence/display/DORIS/DSIP-006%3A+Refactor+IO+stack
Use case
- No response*
Related issues
- No response*
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
1条答案
按热度按时间gc0ot86w1#
Currently, Doris IO related code dependencies:
Rowset -> BlockManager -> ReadableBlock/WritableBlock -> Env
Scanner -> FileReader -> Env/S3Client/HDFSFS
FileResultWriter -> FileWriter -> Env/S3Client/HDFSFS
Expected dependencies:
Rowset -> FileSystem -> ReadStream/WriteStream (different Rowset may use different FileSystem backend)
Scanner -> ReadStream
FileResultWriter -> WriteStream
FileSystem provides APIs for directory and file management, and may manage the file cache in the future.
ReadStream/WriteStream may contain buffer and prefetch data in parallel.