Environment
- Spring Cloud Alibaba 2.2.3.RELEASE
- Dubbo version: 2.7.8
- Java version: 1.8
当前有如下问题:
10.20.2.18:20880为阿里云k8s上建立的私网集群负载均衡服务
容器副本两台:172.17.0.190和172.17.0.191
容器(PodIP:172.17.0.190)无法访问 负载均衡:10.20.2.18:20880(轮询到自身172.17.0.190:20880时)
容器(PodIP:172.17.0.190)可以访问 负载均衡:10.20.2.18:20880(轮询到172.17.0.191:20880时)
容器(PodIP:172.17.0.191)无法访问 负载均衡:10.20.2.18:20880(轮询到自身172.17.0.191:20880时)
容器(PodIP:172.17.0.191)可以访问 负载均衡:10.20.2.18:20880(轮询到172.17.0.190:20880时)
具体现象:任何一台容器 消费dubbo时候,一次成功,一次失败。
出现这个问题的原因是,nacos配置直接用 负载均衡ip做了注册:
spring.cloud.nacos.discovery.server-addr=xx.xx.xx.xx:8848
spring.cloud.nacos.discovery.ip=10.20.2.18
spring.cloud.nacos.discovery.port=8080
dubbo注册服务时使用PodIP注册(172.17.0.190:20880和172.17.0.191:20880),消费的时候却使用了负载均衡ip,即10.20.2.18:20880
网上搜了一圈,好像K8S容器自身与 负载均衡解析到自身时是不通的。
为什么非要指定负载均衡ip?因为公司环境相对复杂,服务器资源在多个阿里云账号下,虽然拉了内网专线
但不属于同一个K8S内网,因此不同账号下的容器内部Pod网络不通(即172网段),需要做负载均衡转发。
因此想到这种解决方案,如果纯spring cloud的话,没有问题,恰恰dubbo和springcloud注册的是同一个服务名(application.name),且指定了同一个ip:10.20.2.18
理想状态,dubbo 服务纯粹仅供系统内部调用,因此注册使用PodIP即可。而对外暴露feign调用的话,使用负载均衡ip做解析
但是目前就是 dubbo虽然注册上去是172网段,但是消费的时候,全部使用10.20.2.18负载均衡网段。
想来想去,这套架构,最合适的方案就是:系统与系统间feign用 负载均衡ip(10.xx),系统内部dubbo用 容器内部网段(172.xx)消费
请问大牛,想要指定dubbo消费网段要怎么做?
或者说,dubbo的服务与 cloud的服务区分开,也能间接解决此问题!
Steps to reproduce this issue
- 将nacos注册ip和端口指定到k8s负载均衡ip上
- 任意一个consumer消费provider,副本数为2,一次可用,一次不可用
- 检测的方法还可以进容器,直接 curl http://负载均衡ip:8080/xxx ,一次通,一次不通
Pls. provide [GitHub address] to reproduce this issue.
Expected Result
172.17.0.191 消费 172.17.0.190:20880或172.17.0.191:20880
Actual Result
172.17.0.191 消费 10.20.2.18:20880
一次通,一次拒绝
2021-04-30 14:47:57.689 INFO [dubbo-client-idleCheck-thread-1] ReconnectTimerTask - [DUBBO] Initial connection to HeaderExchangeClient [channel=org.apache.dubbo.remoting.transport.netty4.NettyClient [172.17.0.190:0 -> /10.20.2.19:20880]], dubbo version: 2.7.8, current host: 172.17.0.190
2021-04-30 14:47:57.694 ERROR [dubbo-client-idleCheck-thread-1] ReconnectTimerTask - [DUBBO] Fail to connect to HeaderExchangeClient [channel=org.apache.dubbo.remoting.transport.netty4.NettyClient [172.17.0.190:0 -> /10.20.2.19:20880]], dubbo version: 2.7.8, current host: 172.17.0.190
org.apache.dubbo.remoting.RemotingException: client(url: dubbo://10.20.2.19:20880/com.joyoung.wsc.product.api.service.ProductService?anyhost=true&application=wsc-gateway&check=false&codec=dubbo&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&heartbeat=60000&init=false&interface=com.joyoung.wsc.product.api.service.ProductService&loadbalance=roundrobin&metadata-type=remote&methods=saveSku,updateProduct,getById,addProduct,listProduct,listH5mProduct,deleteSku,deleteProduct,updateProductStatus,getProductById&pid=1&qos.enable=false®ister.ip=172.17.0.190&release=2.7.8&remote.application=wsc-product&retries=0&revision=1.0-SNAPSHOT&service.filter=-exception&side=consumer&sticky=false&timeout=60000) failed to connect to server /10.20.2.19:20880, error message is:Connection refused: /10.20.2.19:20880
at org.apache.dubbo.remoting.transport.netty4.NettyClient.doConnect(NettyClient.java:169)
at org.apache.dubbo.remoting.transport.AbstractClient.connect(AbstractClient.java:191)
at org.apache.dubbo.remoting.transport.AbstractClient.reconnect(AbstractClient.java:247)
at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeClient.reconnect(HeaderExchangeClient.java:166)
at org.apache.dubbo.remoting.exchange.support.header.ReconnectTimerTask.doTask(ReconnectTimerTask.java:49)
at org.apache.dubbo.remoting.exchange.support.header.AbstractTimerTask.run(AbstractTimerTask.java:87)
at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:648)
at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:727)
at org.apache.dubbo.common.timer.HashedWheelTimer$Worker.run(HashedWheelTimer.java:449)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /10.20.2.19:20880
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
1条答案
按热度按时间bpzcxfmw1#
dubbo服务单独分组,或者单独分名称空间
or