Search before asking

• I had searched in the issues and found no similar issues.

Description

The current version( #8561 ) of vectorized stream load is using skip list to aggregate values, which is a row-structured constant ordered data result.

Considering that we just do pre-aggregation in memtable and queries will not go to memtable, this implementation of maintaining constant order is too expensive.

I plan to refactor this part of the code and replace the existing skip list with other solutions based on pr #8561.

Solution

Solution 1:
First, sort of incoming-data-block.
Then, merge the sorted-data-block.
Then, append merged-data-block to final-block.
At last, do a finalize(sort + merge) of the final-block to flush.

Solution 2:
First, aggregate the incoming-data-block by hash table.
At last, sort the whole aggregated-block.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct