我使用mongo spark连接器2.4.1。mongo db和spark的最新版本。mongodb和spark在不同的docker容器中工作
使用:
...
sc.read.format("mongo").load()
sc.count()
数据量巨大,约30毫升
但是90%的时间里,每10秒就会产生很多次Spark。尝试这样做:
20/05/17 20:38:23 DEBUG cluster: Checking status of mongo_db:27017
20/05/17 20:38:23 DEBUG command: Sending command '{"ismaster": 1, "$db": "admin"}' with request id 60 to database admin on connection [connectionId{localValue:3, serverValue:9}] to server mongo_db:27017
20/05/17 20:38:23 DEBUG command: Execution of command with request id 60 completed successfully in 0.91 ms on connection [connectionId{localValue:3, serverValue:9}] to server mongo_db:27017
20/05/17 20:38:23 DEBUG cluster: Updating cluster description to {type=STANDALONE, servers=[{address=mongo_db:27017, type=STANDALONE, roundTripTime=1.3 ms, state=CONNECTED}]
这是什么意思?如何解决?
暂无答案!
目前还没有任何答案,快来回答吧!