我正在尝试用ssl将spark连接到elasticsearch。
设置

Spark 2.4.0 from CDH 6.3.2 (Cloudera)
ElasticSearch 7.6.1 (Open Distro)
elasticsearch-hadoop-7.6.1.jar

考虑到

1) I already managed to authenticate logstash with SSL and pkcs12 keystore manually created
2) Connexion Spark to ES works without security

此处提供：

spark.es.nodes=node1
spark.es.port=9200
spark.es.net.ssl=true
spark.es.net.ssl.keystore.location= ===> See below what i tried
spark.es.net.ssl.keystore.type=PKCS12
spark.es.net.ssl.cert.allow.self.signed=true
spark.es.net.http.auth.user=admin
spark.es.net.http.auth.pass=admin
spark.es.nodes.wan.only=false  //tried true

做

spark.read.format("org.elasticsearch.spark.sql")
    .option("es.query", "?q=*:*")
    .load("spark/docs")
    .show

====================================================

使用spark.es.net.ssl.keystore.location尝试的文件系统值（在所有节点上复制admin.pkcs12之后）

file:///PATH/certs/admin.pkcs12

错误：

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
  ... elided
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - Get Key failed: null
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:175)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.getSSLContext(SSLSocketFactory.java:160)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSocket(SSLSocketFactory.java:129)
  at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
  at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
  at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
  at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
  at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
  at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.doExecute(CommonsHttpTransport.java:685)
  at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:664)
  at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:432)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:388)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:392)
  at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:168)
  at org.elasticsearch.hadoop.rest.RestClient.mainInfo(RestClient.java:745)
  at org.elasticsearch.hadoop.rest.InitializationUtils.discoverClusterInfo(InitializationUtils.java:330)
  ... 61 more
Caused by: java.security.UnrecoverableKeyException: Get Key failed: null
  at sun.security.pkcs12.PKCS12KeyStore.engineGetKey(PKCS12KeyStore.java:435)
  at java.security.KeyStore.getKey(KeyStore.java:1023)
  at sun.security.ssl.SunX509KeyManagerImpl.<init>(SunX509KeyManagerImpl.java:133)
  at sun.security.ssl.KeyManagerFactoryImpl$SunX509.engineInit(KeyManagerFactoryImpl.java:70)
  at javax.net.ssl.KeyManagerFactory.init(KeyManagerFactory.java:256)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.loadKeyManagers(SSLSocketFactory.java:217)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:173)
  ... 78 more
Caused by: java.lang.NullPointerException
  at sun.security.pkcs12.PKCS12KeyStore.engineGetKey(PKCS12KeyStore.java:374)
  ... 84 more

====================================================

我用777权限将一个密钥库a valid admin.pkcs12复制到hdfs=>/user/company/（在我写的时候，它是不是像ssh一样过于宽松？）

//returns true
FileSystem.get(spark.sparkContext.hadoopConfiguration).exists(new Path("hdfs://namenode:8020/user/company/admin.pkcs12"))

使用spark.es.net.ssl.keystore.location尝试的hdfs值

hdfs:///namenode:8020/user/company/admin.pkcs12
hdfs://namenode:8020/user/company/admin.pkcs12
/user/company/admin.pkcs12

错误：

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
  ... elided
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - Expected to find keystore file at [...] but was unable to. Make sure that it is available on the classpath, or if not, that you have specified a valid URI.
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:175)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.getSSLContext(SSLSocketFactory.java:160)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSocket(SSLSocketFactory.java:129)
  at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
  at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
  at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
  at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
  at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
  at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.doExecute(CommonsHttpTransport.java:685)
  at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:664)
  at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:116)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:432)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:388)
  at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:392)
  at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:168)
  at org.elasticsearch.hadoop.rest.RestClient.mainInfo(RestClient.java:745)
  at org.elasticsearch.hadoop.rest.InitializationUtils.discoverClusterInfo(InitializationUtils.java:330)
  ... 61 more
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Expected to find keystore file at [...] but was unable to. Make sure that it is available on the classpath, or if not, that you have specified a valid URI.
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.loadKeyStore(SSLSocketFactory.java:195)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.loadKeyManagers(SSLSocketFactory.java:215)
  at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:173)

我也试过jks。我错过了什么？