curator简介

x33g5p2x  于2021-03-14 发布在 ElasticSearch  
字(3.4k)|赞(0)|评价(0)|浏览(286)

curator简介

curator用于管理es索引的生命周期。

主要目的:规划和管理ES的索引。支持常见操作:创建、删除、合并、reindex、快照等操作。

es地址:https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html

Github地址:https://github.com/elastic/curator

镜像地址:https://hub.docker.com/r/bobrik/curator

安装curator

配置yum源/etc/yum.repos.d/curator.repo

[curator-5]
name=CentOS/RHEL 7 repository for Elasticsearch Curator 5.x packages
baseurl=https://packages.elastic.co/curator/5/centos/7
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

通过yum安装

$ yum install elasticsearch-curator

配置curator

configfile配置参考官方

https://www.elastic.co/guide/en/elasticsearch/client/curator/5.7/configfile.html

---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
    - 192.168.31.10
    - 192.168.31.11
    - 192.168.31.12
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 600
  master_only: False

logging:
  loglevel: DEBUG
  logfile: #/home/curator/curator.log
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']

action.yml配置包含四个部分

  • action 操作
  • description 描述
  • options 参数
  • filters 匹配方式

示例

删除索引

actions:
  1:
    action: delete_indices
    description: "删除过期索引"
    options:
      # action开关,为True表示不执行这个action,默认False
      disable_action: False
      # 如果为True,则在filters空列表时,继续下一个action处理而不是退出程序。
      ignore_empty_list: True
      # 发现错误后,继续执行下一个索引操作,默认False
      continue_if_exception: True
    filters:
    # 是否排除隐藏索引,如.kibana
    - filtertype: kibana
      exclude: True
    # 是否排除open状态的索引
    - filtertype: opened
      exclude: True
    # 处理30天前的索引
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 30
      exclude: False

分配调度

actions:
  1:
    action: allocation
    description: "Apply shard allocation filtering rules to the specified indices"
    options:
      key: box_type
      value: warm
      allocation_type: require
      wait_for_completion: true
      continue_if_exception: true
      disable_action: false
    filters:
    - filtertype: pattern
      kind: prefix
      value: traefik-
      exclude: False
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 6
      exclude: False

actions:
  1:
    action: allocation
    description: "Apply shard allocation filtering rules to the specified indices"
    options:
      disable_action: False
      key: box_type
      value: warm
      allocation_type: require
      wait_for_completion: False
      # 发现错误后,继续执行下一个索引操作
      continue_if_exception: True
      ignore_empty_list: True
    filters:
    - filtertype: pattern
      kind: prefix
      value: nginx_kong-
      exclude: False
    - filtertype: period
      source: name
      timestring: '%Y.%m.%d'
      unit: days
      range_from: -22
      range_to: -22
      exclude: False

段落合并

actions:
  1:
    action: delete_indices
    description: "删除过期索引"
    options:
      # action开关,为True表示不执行这个action,默认False
      disable_action: False
      # 如果为True,则在filters空列表时,继续下一个action处理而不是退出程序。
      ignore_empty_list: True
      # 发现错误后,继续执行下一个索引操作,默认False
      continue_if_exception: True
    filters:
    # 是否排除隐藏索引,如.kibana
    - filtertype: kibana
      exclude: True
    # 是否排除open状态的索引
    - filtertype: opened
      exclude: True
    # 处理30天前的索引
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 30
      exclude: False

curator命令行操作

1$ curator --help
2Usage: curator [OPTIONS] ACTION_FILE
3  Curator for Elasticsearch indices.
4  See http://elastic.co/guide/en/elasticsearch/client/curator/current
5Options:
6  --config PATH  Path to configuration file. Default: ~/.curator/curator.yml
7  --dry-run      Do not perform any changes.
8  --version      Show the version and exit.
9  --help         Show this message and exit.

核心:

  • 配置文件config.yml:配置要连接的ES地址、日志配置、日志级别等;
  • 执行文件action.yml: 配置要执行的操作(可批量)、配置索引的格式(前缀匹配、正则匹配方式等)

curator适用场景

最重要的是:

  • 仅以删除操作为例:curator可以非常简单地删除x天后的索引的前提是:索引命名要遵循特定的命名模式——如:以天为命名的索引:logstash_2018.04.05。
  • 命名模式需要和action.yml中的delete_indices下的timestring对应。

相关文章

热门文章

更多