incubator-doris [bug]一个简单查询,会时不时unknown error

vhipe2zx  于 2022-04-22  发布在  Java
关注(0)|答案(5)|浏览(286)

Describe the bug

一个简单查询:
select id from 表 limit 1

在代码中:
while(true){
JDBC 执行 SQL
}

会不定期出现:
Exception in thread "main" com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown error

在fe.warn.log中看到的日志是:
2020-09-22 17:21:13,368 WARN 24043 [StmtExecutor.execute():306] execute Exception
java.util.ConcurrentModificationException: null
at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211) ~[?:1.8.0_252]
at java.util.TreeMap$KeyIterator.next(TreeMap.java:1265) ~[?:1.8.0_252]
at org.apache.doris.common.util.RuntimeProfile.printChildCounters(RuntimeProfile.java:235) ~[palo-fe.jar:?]
at org.apache.doris.common.util.RuntimeProfile.prettyPrint(RuntimeProfile.java:212) ~[palo-fe.jar:?]
at org.apache.doris.common.util.RuntimeProfile.prettyPrint(RuntimeProfile.java:219) ~[palo-fe.jar:?]
at org.apache.doris.common.util.RuntimeProfile.prettyPrint(RuntimeProfile.java:219) ~[palo-fe.jar:?]
at org.apache.doris.common.util.RuntimeProfile.prettyPrint(RuntimeProfile.java:219) ~[palo-fe.jar:?]
at org.apache.doris.common.util.RuntimeProfile.prettyPrint(RuntimeProfile.java:219) ~[palo-fe.jar:?]
at org.apache.doris.qe.StmtExecutor.writeProfile(StmtExecutor.java:342) ~[palo-fe.jar:?]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:244) ~[palo-fe.jar:?]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:184) ~[palo-fe.jar:?]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:316) ~[palo-fe.jar:?]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:501) ~[palo-fe.jar:?]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-09-22 17:21:13,385 WARN 24043 [ConnectProcessor.processOnce():491] Null packet received from network. remote: XXX.XXX.XXX.74:38188
2020-09-22 17:21:13,385 WARN 24043 [ReadListener.lambda$handleEvent$0():58] Exception happened in one session(org.apache.doris.mysql.nio.NConnectContext@4f53ffef).
java.io.IOException: Error happened when receiving packet.
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:492) ~[palo-fe.jar:?]
at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]

补充:
fe配置:
max_routine_load_job_num = 270
max_layout_length_per_row = 100000000
max_routine_load_task_num_per_be = 27

label_keep_max_second = 3600
history_job_keep_max_second = 3600
label_clean_interval_second = 3600
max_running_txn_num_per_db = 300

audit_log_delete_age = 7d
sys_log_delete_age = 7d

query_colocate_join_memory_limit_penalty_factor = 1

thrift_server_max_worker_threads = 6144

be配置:
routine_load_thread_pool_size = 27
sys_log_roll_num = 1

base_compaction_num_threads_per_disk = 3
base_compaction_check_interval_seconds = 10
cumulative_compaction_check_interval_seconds = 1
cumulative_compaction_skip_window_seconds = 10
min_compaction_failure_interval_sec = 10
cumulative_compaction_num_threads_per_disk = 3

10个be节点,20分桶,无分区

qmelpv7a

qmelpv7a1#

不是很理解一个查询的SQL,为什么会出现一致性修改错误,一般不都是对iterator遍历的时候做删除,会出这种错误嘛

cedebl8k

cedebl8k2#

再次补充:
通过be.log,在查询失败附近,会发现:
INFO 393 [QeProcessorImpl.reportExecStatus():118] ReportExecStatus() runtime error, query TUniqueId(hi:-5892633407530515572, lo:-7549245074990846607) does not exist

hfyxw5xn

hfyxw5xn3#

再次补充,只有使用了limit 的简单查询,且limit起到了作用(比如limit 99999999999999999就是不起作用的),就会出现这个异常,但是在测试环境却没有见过这个异常

cwtwac6a

cwtwac6a4#

@li-ang-666 Hi, @HangyuanLiu has fixed this bug.

@HangyuanLiu Hi, please submit a PR when you have time, thanks!

yyyllmsg

yyyllmsg5#

@li-ang-666 Hi, @HangyuanLiu has fixed this bug.

@HangyuanLiu Hi, please submit a PR when you have time, thanks!

@kangkaisen 0.13.9 , 这个bug修复好了嘛

相关问题