dcos上的spark：使调度程序从集群外部可访问

9fkzdhlc 于 2021-06-26 发布在 Mesos

关注(0)|答案(1)|浏览(355)

我们已经安装了星火服务 marathon 目录 DCOS 集群。
服务的json配置如下：

{
  "service": {
    "name": "spark",
    "cpus": 1,
    "mem": 1024,
    "role": "*",
    "service_account": "",
    "service_account_secret": "",
    "user": "root",
    "docker-image": "mesosphere/spark:2.3.1-2.2.1-2-hadoop-2.6",
    "log-level": "INFO",
    "spark-history-server-url": "http://internal-tfef5a-int-master-elb-1145533197.eu-east-1.elb.amazonaws.com/service/spark-history",
    "UCR_containerizer": false,
    "use_bootstrap_for_IP_detect": false
  },
  "hdfs": {
    "config-url": "http://api.hdfs.marathon.l4lb.thisdcos.directory/v1/endpoints"
  }
}

鉴于 marathon ui不允许直接编辑服务端口，那么公开dispatcher的正确方法是什么 spark-submit 从他/她自己的工作站？