mongo spark连接器减速操作多次检查mongo\ db的状态

bsxbgnwa  于 2021-05-27  发布在  Spark
关注(0)|答案(0)|浏览(339)

我使用mongo spark连接器2.4.1。mongo db和spark的最新版本。mongodb和spark在不同的docker容器中工作
使用:

...
sc.read.format("mongo").load()
sc.count()

数据量巨大,约30毫升
但是90%的时间里,每10秒就会产生很多次Spark。尝试这样做:

20/05/17 20:38:23 DEBUG cluster: Checking status of mongo_db:27017
20/05/17 20:38:23 DEBUG command: Sending command '{"ismaster": 1, "$db": "admin"}' with request id 60 to database admin on connection [connectionId{localValue:3, serverValue:9}] to server mongo_db:27017
20/05/17 20:38:23 DEBUG command: Execution of command with request id 60 completed successfully in 0.91 ms on connection [connectionId{localValue:3, serverValue:9}] to server mongo_db:27017
20/05/17 20:38:23 DEBUG cluster: Updating cluster description to  {type=STANDALONE, servers=[{address=mongo_db:27017, type=STANDALONE, roundTripTime=1.3 ms, state=CONNECTED}]

这是什么意思?如何解决?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题