fluentd sidecar无法将日志发送到elasticsearch:不赞成删除类型在批量请求中指定类型

unhi4e5o  于 2021-06-14  发布在  ElasticSearch
关注(0)|答案(1)|浏览(721)

我配置了一个sidecar容器来收集我的应用程序容器的日志。
我首先测试了我的 fleunt.conf 使用以下配置:

<source>
    @type forward
    bind "127.0.0.1"
    port 24224
    <parse>
      @type json
    </parse>
  </source>
  <match app.default>
    @type stdout
  </match>

当我跟踪日志时,效果很好 fluentd 容器在我的吊舱里使用 kubectl ,我可以看到json格式的应用程序日志。
现在,我想把日志发送到 elasticsearch . 这是我的 fluent.conf 同样地:

<source>
    @type forward
    bind "127.0.0.1"
    port 24224
    <parse>
      @type json
    </parse>
  </source>
  <match app.default>
    @type elasticsearch
    host "elasticsearch_host"
    port 9200
    index_name "app-log"
    user "log_user"
    password xxxxxx
  </match>

fluentd容器在我的pod中运行良好,但我无法定义在中配置的索引 fluetn.conf 在Kibana。看起来索引模式没有加载到 elasticsearch .
这个 fluentd 容器显示以下警告:

2020-10-22 12:31:10 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2020-10-22 12:31:10 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '4.0.0'
2020-10-22 12:31:10 +0000 [info]: gem 'fluentd' version '1.10.4'
2020-10-22 12:31:12 +0000 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    bind "127.0.0.1"
    port 24224
    <parse>
      @type json
    </parse>
  </source>
  <match app.default>
    @type elasticsearch
    host "elasticsearch_host"
    port 9200
    index_name "app-log"
    user "log_user"
    password xxxxxx
  </match>
</ROOT>
2020-10-22 12:31:12 +0000 [info]: starting fluentd-1.10.4 pid=8 ruby="2.5.8"
2020-10-22 12:31:12 +0000 [info]: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "-p", "/fluentd/plugins", "--under-supervisor"]
2020-10-22 12:31:16 +0000 [info]: adding match pattern="app.default" type="elasticsearch"
2020-10-22 12:31:17 +0000 [warn]: #0 Detected ES 7.x: `_doc` will be used as the document `_type`.
2020-10-22 12:31:17 +0000 [info]: adding source type="forward"
2020-10-22 12:31:17 +0000 [warn]: section <parse> is not used in <source> of forward plugin
2020-10-22 12:31:17 +0000 [info]: #0 starting fluentd worker pid=22 ppid=8 worker=0
2020-10-22 12:31:17 +0000 [info]: #0 listening port port=24224 bind="127.0.0.1"
2020-10-22 12:31:17 +0000 [info]: #0 fluentd worker is now running worker=0
warning: 299 Elasticsearch-7.5.0-e9ccaed468e2fac2275a3761849cbee64b39519f "[types removal] Specifying types in bulk requests is deprecated."
warning: 299 Elasticsearch-7.5.0-e9ccaed468e2fac2275a3761849cbee64b39519f "[types removal] Specifying types in bulk requests is deprecated."
warning: 299 Elasticsearch-7.5.0-e9ccaed468e2fac2275a3761849cbee64b39519f "[types removal] Specifying types in bulk requests is deprecated."

我该如何摆脱这个警告 "[types removal] Specifying types in bulk requests is deprecated." ? 是否有其他配置 fluent.conf 能帮我度过难关吗?
我试过用 suppress_type_name true 但它没有工作。

ht4b089n

ht4b089n1#

支持 suppress_type_name true仅随插件版本4.0.10一起添加:https://github.com/uken/fluent-plugin-elasticsearch/blob/master/history.md#4010.
根据日志,您正在运行4.0.0,因此升级和设置配置中的标志应该可以消除警告。警告不应影响批量请求。

相关问题