在emr中为presto设置配置单元配置属性

ycggw6v2  于 2021-06-26  发布在  Hive
关注(0)|答案(2)|浏览(319)

我正在尝试更新内部的配置单元连接器属性 /etc/presto/conf/catalog/hive.properties . 我补充说:

hive.s3.max-client-retries=50
hive.s3.max-error-retries=50
hive.s3.max-connections=500
hive.s3.connect-timeout=5m
hive.s3.socket-timeout=5m

但由于某些原因,这些都不被接受。presto服务器重新启动失败,错误如下:

2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   UNUSED PROPERTIES
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.connect-timeout
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-client-retries
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-connections
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-error-retries
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.socket-timeout
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   
2018-04-16T19:34:26.587Z    ERROR   main    com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:

1) Configuration property 'hive.s3.connect-timeout' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

2) Configuration property 'hive.s3.max-client-retries' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

3) Configuration property 'hive.s3.max-connections' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

4) Configuration property 'hive.s3.max-error-retries' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

5) Configuration property 'hive.s3.socket-timeout' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

5 errors
2ul0zpep

2ul0zpep1#

正如piotr提到的,emr从emr-5.12.0开始将默认的文件系统实现从prestos3filesystem更改为emrfs。prestos3filesystem属性将不可用,但您可以像配置任何其他应用程序一样配置emrfs。请参阅上的“emrfs and prestos3filesystem configuration”https://docs.aws.amazon.com/emr/latest/releaseguide/emr-presto-considerations.html 更多信息。

0ejtzxu1

0ejtzxu12#

这些属性适用于 hive.s3-file-system-type 设置为 PRESTO . 当它是例如。 EMRFS ,这些配置属性不适用,设置它们会导致您观察到的错误。

相关问题