Error happens when set nacos.server-addr to be a domain name

ymzxtsji  于 2022-11-13  发布在  Nacos
关注(0)|答案(9)|浏览(322)
  • Step1:

I use helm to release the nacos server, and expose our domain name like this: nacos.dev.company.com
here is the ingress rule:

http:
        paths:
        - pathType: Prefix
          path: /
          backend:
            service:
              name: nacos-cs
              port:
                number: 8848

By the way, the domain name is just the internal domain name, protected by VPN.

  • Step2:

My configuration is:

spring:
  cloud:
    nacos:
      server-addr: nacos.dev.company.com
  • Step3:

Run springboot application, I got this error:
[ERROR] Server check fail, please check server nacos.dev.company.com, port 1080 is available , error ={}

Debug info:

public Connection connectToServer(ServerInfo serverInfo) {
        try {
            if (grpcExecutor == null) {
                this.grpcExecutor = createGrpcExecutor(serverInfo.getServerIp());
            }
            int port = serverInfo.getServerPort() + rpcPortOffset();  // maybe affected by this logic 
         ...

Question
Cann't the configuration of server-addr be a domain name ?

vlju58qv

vlju58qv1#

If your server-addr is a domain name and your nacos server port is not 80 , you need to set the port explicitly. For example the following example:

spring:
  cloud:
    nacos:
      server-addr: nacos.dev.company.com:8848
1mrurvl1

1mrurvl12#

If your server-addr is a domain name and your nacos server port is not 80 , you need to set the port explicitly. For example the following example:

spring:
  cloud:
    nacos:
      server-addr: nacos.dev.company.com:8848

well, the nacos server port is 80, not 8848, because it's routed by k8s ingress. The domain name is a general design in prod, and it's not supported by the nacos client sdk, I'm confused by the design of nacos client. Why not make the port configurable ? 8848 is so hard code.

By the way, are you sure the design from https://github.com/nacos-group/nacos-k8s/tree/master/helm is ready for prod useage ?

slhcrj9b

slhcrj9b3#

Make sure your 1080 port opened

2w2cym1i

2w2cym1i4#

Make sure your 1080 port opened

Would you please show me how to make port 1080 open in k8s ingress ? Is there any note in https://github.com/nacos-group/nacos-k8s/tree/master/helm ? Or any instructions in https://nacos.io/en-us/docs/v2/quickstart/quick-start-kubernetes.html

7rtdyuoh

7rtdyuoh5#

The helm has open 9848(if main port is 80, the port should be 1080).

If you access nacos by ingress, you should open ingress port 9848(1080). For most of application, they should deploy same k8s inner, so no need open 9848(1080) in ingress.

slmsl1lt

slmsl1lt6#

@shihui-tang
First of all, I must say that I am not very much in favor of the production going to open the registration center to the extranet.

Secondly, although I don't really agree with it, if you really want to do it, you can also try to do it in the following way.
For example by Nginx Ingress

  1. First create a ConfigMap for the specified port
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-tcp-services
  namespace: nacos    # Note that the namespace here needs to be modified according to your own it
data:
  8848: "nacos/nacos-headless:8848"  
  9848: "nacos/nacos-headless:9848"
  1. Modify your Nginx Ingress deployment file to add the following startup parameters
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-nginx-controller
  namespace: default
spec:
...
  template:
...
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --tcp-services-configmap=nacos/nacos-tcp-services
...
  1. Then we need to expose port 8848/9848 in the Service defined for the Ingress.
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
    - name: nacos-client-grpc-9848
      port: 9848
      targetPort: 9848
      protocol: TCP
    - name: nacos-client-grpc-8848
      port: 8848
      targetPort: 8848
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  1. When you're done with this you can remove the ingress configuration generated by the previous helm, and Modify the address of the registration center as follows.
spring:
  cloud:
    nacos:
      server-addr: nacos.dev.company.com:8848

BTW, If you want to understand the specific meaning of the above configuration, you can go to the following exposing-tcp-udp-services

Thanks

eivgtgni

eivgtgni7#

@paderlol Thanks for your help. I will try it.
BTW, Maybe my comment is not clear. We will not expose our nacos to extranet. But we want our internal service access nacos by domain name, not :

c6ubokkw

c6ubokkw8#

Definitely, you can do this, but if you expose the port at 443, you also need to expose the port at 1434, which is the port on which the client communicates with the server Grpc.

apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-tcp-services
  namespace: nacos    # Note that the namespace here needs to be modified according to your own it
data:
  443: "nacos/nacos-headless:8848"  
  1443: "nacos/nacos-headless:9848"
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 8848
      protocol: TCP
    - name: nacos-client-grpc
      port: 1443
      targetPort: 9848
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
dgsult0t

dgsult0t9#

@paderlol Here is new issue about port: 443, it can not be overwriten

controller.go:395] Port 443 cannot be used for TCP stream services. It is reserved for the Ingress controller.

Would you please provide the whole example for k8s yamls ? Thanks.

相关问题