dubbo注册服务和cloud服务如何在nacos上区分开?

jdg4fx2g  于 2022-11-13  发布在  Nacos
关注(0)|答案(1)|浏览(220)

Environment

  • Spring Cloud Alibaba 2.2.3.RELEASE
  • Dubbo version: 2.7.8
  • Java version: 1.8

当前有如下问题:
10.20.2.18:20880为阿里云k8s上建立的私网集群负载均衡服务

容器副本两台:172.17.0.190和172.17.0.191
容器(PodIP:172.17.0.190)无法访问 负载均衡:10.20.2.18:20880(轮询到自身172.17.0.190:20880时)
容器(PodIP:172.17.0.190)可以访问 负载均衡:10.20.2.18:20880(轮询到172.17.0.191:20880时)

容器(PodIP:172.17.0.191)无法访问 负载均衡:10.20.2.18:20880(轮询到自身172.17.0.191:20880时)
容器(PodIP:172.17.0.191)可以访问 负载均衡:10.20.2.18:20880(轮询到172.17.0.190:20880时)

具体现象:任何一台容器 消费dubbo时候,一次成功,一次失败。

出现这个问题的原因是,nacos配置直接用 负载均衡ip做了注册:

spring.cloud.nacos.discovery.server-addr=xx.xx.xx.xx:8848
spring.cloud.nacos.discovery.ip=10.20.2.18
spring.cloud.nacos.discovery.port=8080

dubbo注册服务时使用PodIP注册(172.17.0.190:20880和172.17.0.191:20880),消费的时候却使用了负载均衡ip,即10.20.2.18:20880

网上搜了一圈,好像K8S容器自身与 负载均衡解析到自身时是不通的。

为什么非要指定负载均衡ip?因为公司环境相对复杂,服务器资源在多个阿里云账号下,虽然拉了内网专线

但不属于同一个K8S内网,因此不同账号下的容器内部Pod网络不通(即172网段),需要做负载均衡转发。

因此想到这种解决方案,如果纯spring cloud的话,没有问题,恰恰dubbo和springcloud注册的是同一个服务名(application.name),且指定了同一个ip:10.20.2.18

理想状态,dubbo 服务纯粹仅供系统内部调用,因此注册使用PodIP即可。而对外暴露feign调用的话,使用负载均衡ip做解析

但是目前就是 dubbo虽然注册上去是172网段,但是消费的时候,全部使用10.20.2.18负载均衡网段。

想来想去,这套架构,最合适的方案就是:系统与系统间feign用 负载均衡ip(10.xx),系统内部dubbo用 容器内部网段(172.xx)消费

请问大牛,想要指定dubbo消费网段要怎么做?
或者说,dubbo的服务与 cloud的服务区分开,也能间接解决此问题!

Steps to reproduce this issue

  1. 将nacos注册ip和端口指定到k8s负载均衡ip上

  1. 任意一个consumer消费provider,副本数为2,一次可用,一次不可用
  2. 检测的方法还可以进容器,直接 curl http://负载均衡ip:8080/xxx ,一次通,一次不通

Pls. provide [GitHub address] to reproduce this issue.

Expected Result

172.17.0.191 消费 172.17.0.190:20880或172.17.0.191:20880

Actual Result

172.17.0.191 消费 10.20.2.18:20880
一次通,一次拒绝

2021-04-30 14:47:57.689 INFO  [dubbo-client-idleCheck-thread-1] ReconnectTimerTask -  [DUBBO] Initial connection to HeaderExchangeClient [channel=org.apache.dubbo.remoting.transport.netty4.NettyClient [172.17.0.190:0 -> /10.20.2.19:20880]], dubbo version: 2.7.8, current host: 172.17.0.190
2021-04-30 14:47:57.694 ERROR [dubbo-client-idleCheck-thread-1] ReconnectTimerTask -  [DUBBO] Fail to connect to HeaderExchangeClient [channel=org.apache.dubbo.remoting.transport.netty4.NettyClient [172.17.0.190:0 -> /10.20.2.19:20880]], dubbo version: 2.7.8, current host: 172.17.0.190
org.apache.dubbo.remoting.RemotingException: client(url: dubbo://10.20.2.19:20880/com.joyoung.wsc.product.api.service.ProductService?anyhost=true&application=wsc-gateway&check=false&codec=dubbo&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&heartbeat=60000&init=false&interface=com.joyoung.wsc.product.api.service.ProductService&loadbalance=roundrobin&metadata-type=remote&methods=saveSku,updateProduct,getById,addProduct,listProduct,listH5mProduct,deleteSku,deleteProduct,updateProductStatus,getProductById&pid=1&qos.enable=false&register.ip=172.17.0.190&release=2.7.8&remote.application=wsc-product&retries=0&revision=1.0-SNAPSHOT&service.filter=-exception&side=consumer&sticky=false&timeout=60000) failed to connect to server /10.20.2.19:20880, error message is:Connection refused: /10.20.2.19:20880
	at org.apache.dubbo.remoting.transport.netty4.NettyClient.doConnect(NettyClient.java:169)
	at org.apache.dubbo.remoting.transport.AbstractClient.connect(AbstractClient.java:191)
	at org.apache.dubbo.remoting.transport.AbstractClient.reconnect(AbstractClient.java:247)
	at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeClient.reconnect(HeaderExchangeClient.java:166)
	at org.apache.dubbo.remoting.exchange.support.header.ReconnectTimerTask.doTask(ReconnectTimerTask.java:49)
	at org.apache.dubbo.remoting.exchange.support.header.AbstractTimerTask.run(AbstractTimerTask.java:87)
	at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:648)
	at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:727)
	at org.apache.dubbo.common.timer.HashedWheelTimer$Worker.run(HashedWheelTimer.java:449)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /10.20.2.19:20880
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)
bpzcxfmw

bpzcxfmw1#

dubbo服务单独分组,或者单独分名称空间

dubbo.registry.group = dubbo

or

dubbo.registry.parameters.namespace = dubbo

相关问题