ElasticSearch -返回唯一值

y4ekin9u 于 5个月前发布在 ElasticSearch

关注(0)|答案(7)|浏览(46)

如何从记录中获取所有languages的值并使其唯一。

记录

PUT items/1
{ "language" : 10 }

PUT items/2
{ "language" : 11 }

PUT items/3
{ "language" : 10 }

字符串

查询

GET items/_search
{ ... }

# => Expected Response
[10, 11]

型
任何帮助将是伟大的。

elasticsearch

来源：https://stackoverflow.com/questions/25465215/elasticsearch-return-unique-values

7条答案

按热度按时间

cetgtptt1#

您可以使用terms aggregation。

{
"size": 0,
"aggs" : {
    "langs" : {
        "terms" : { "field" : "language",  "size" : 500 }
    }
}}

字符串
聚合中的size参数指定要包含在聚合结果中的最大术语数。如果需要所有结果，请将此值设置为大于数据中唯一术语数的值。
搜索将返回如下内容：

{
"took" : 16,
"timed_out" : false,
"_shards" : {
  "total" : 2,
  "successful" : 2,
  "failed" : 0
},
"hits" : {
"total" : 1000000,
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
  "langs" : {
    "buckets" : [ {
      "key" : "10",
      "doc_count" : 244812
    }, {
      "key" : "11",
      "doc_count" : 136794
 
    }, {
      "key" : "12",
      "doc_count" : 32312
       } ]
    }
  }
}

型

赞(0）回复(0）举报 5个月前

bn31dyow2#

我也在为我自己寻找这种解决方案。我在terms aggregation中找到了参考。
因此，根据这一点，以下是适当的解决方案。

{
"aggs" : {
    "langs" : {
        "terms" : { "field" : "language",  
                    "size" : 500 }
    }
}}

字符串
如果你遇到以下错误：

"error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [fastest_method] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
            }
        ]}

型
在这种情况下，您必须在请求中添加“KEYWORD”，如下所示：

{
    "aggs" : {
        "langs" : {
            "terms" : { "field" : "language.keyword",  
                        "size" : 500 }
        }
    }}

型

赞(0）回复(0）举报 5个月前

yks3o0rb3#

如果你想获得第一个文档中每个language字段的唯一值，你可以这样做：

{
 "query": {
    "match_all": {
    }
  },
  "collapse": {
    "field": "language.keyword",
    "inner_hits": {
    "name": "latest",
      "size": 1
    }
  }
}

字符串

赞(0）回复(0）举报 5个月前

j2datikz4#

如果你想得到所有的唯一值，而不需要任何近似值或设置一个幻数（size: 500），那么使用COMPOSITE AGGREGATION（ES 6.5+）。
来自官方文件：

“如果要检索嵌套术语聚合中的所有术语或术语的所有组合**，应使用COMPOSITE AGGREGATION**，它允许对所有可能的术语进行分页，而不是将大小设置为大于术语聚合中字段的基数。术语聚合旨在返回顶部术语，不允许分页。"*

JavaScript中的实现示例：

const ITEMS_PER_PAGE = 1000;

const body =  {
    "size": 0, // Returning only aggregation results: https://www.elastic.co/guide/en/elasticsearch/reference/current/returning-only-agg-results.html
    "aggs" : {
        "langs": {
            "composite" : {
                "size": ITEMS_PER_PAGE,
                "sources" : [
                    { "language": { "terms" : { "field": "language" } } }
                ]
            }
        }
     }
};

const uniqueLanguages = [];

while (true) {
  const result = await es.search(body);

  const currentUniqueLangs = result.aggregations.langs.buckets.map(bucket => bucket.key);

  uniqueLanguages.push(...currentUniqueLangs);

  const after = result.aggregations.langs.after_key;

  if (after) {
      // continue paginating unique items
      body.aggs.langs.composite.after = after;
  } else {
      break;
  }
}

console.log(uniqueLanguages);

字符串

赞(0）回复(0）举报 5个月前

szqfcxe25#

Elasticsearch 1.1+有一个**Cardinality Aggregation**，它会给予你一个唯一的 count，但不是术语本身。
请注意，它实际上是一个近似值，在高基数数据集上，准确性可能会降低，但在我的测试中，它通常非常准确。
您还可以使用precision_threshold参数调整精度。当然，代价是内存使用。
文档中的这张图显示了更高的precision_threshold如何导致更准确的结果。

的数据

赞(0）回复(0）举报 5个月前

ztyzrc3y6#

必须区分两个字段（derivative_id和vehicle_type），并按最便宜的车排序。必须嵌套aggs。

GET /cars/_search
{
  "size": 0,
  "aggs": {
    "distinct_by_derivative_id": {
      "terms": { 
        "field": "derivative_id"
      },
      "aggs": {
        "vehicle_type": {
          "terms": {
            "field": "vehicle_type"
          },
          "aggs": {
            "cheapest_vehicle": {
              "top_hits": {
                "sort": [
                  { "rental": { "order": "asc" } }
                ],
                "_source": { "includes": [ "manufacturer_name",
                  "rental",
                  "vehicle_type" 
                  ]
                },
                "size": 1
              }
            }
          }
        }
      }
    }
  }
}

字符串
测试结果：

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 8,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "distinct_by_derivative_id" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "04",
          "doc_count" : 3,
          "vehicle_type" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "CAR",
                "doc_count" : 2,
                "cheapest_vehicle" : {
                  "hits" : {
                    "total" : {
                      "value" : 2,
                      "relation" : "eq"
                    },
                    "max_score" : null,
                    "hits" : [
                      {
                        "_index" : "cars",
                        "_type" : "_doc",
                        "_id" : "8",
                        "_score" : null,
                        "_source" : {
                          "vehicle_type" : "CAR",
                          "manufacturer_name" : "Renault",
                          "rental" : 89.99
                        },
                        "sort" : [
                          89.99
                        ]
                      }
                    ]
                  }
                }
              },
              {
                "key" : "LCV",
                "doc_count" : 1,
                "cheapest_vehicle" : {
                  "hits" : {
                    "total" : {
                      "value" : 1,
                      "relation" : "eq"
                    },
                    "max_score" : null,
                    "hits" : [
                      {
                        "_index" : "cars",
                        "_type" : "_doc",
                        "_id" : "7",
                        "_score" : null,
                        "_source" : {
                          "vehicle_type" : "LCV",
                          "manufacturer_name" : "Ford",
                          "rental" : 99.99
                        },
                        "sort" : [
                          99.99
                        ]
                      }
                    ]
                  }
                }
              }
            ]
          }
        },
        {
          "key" : "01",
          "doc_count" : 2,
          "vehicle_type" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "CAR",
                "doc_count" : 1,
                "cheapest_vehicle" : {
                  "hits" : {
                    "total" : {
                      "value" : 1,
                      "relation" : "eq"
                    },
                    "max_score" : null,
                    "hits" : [
                      {
                        "_index" : "cars",
                        "_type" : "_doc",
                        "_id" : "1",
                        "_score" : null,
                        "_source" : {
                          "vehicle_type" : "CAR",
                          "manufacturer_name" : "Ford",
                          "rental" : 599.99
                        },
                        "sort" : [
                          599.99
                        ]
                      }
                    ]
                  }
                }
              },
              {
                "key" : "LCV",
                "doc_count" : 1,
                "cheapest_vehicle" : {
                  "hits" : {
                    "total" : {
                      "value" : 1,
                      "relation" : "eq"
                    },
                    "max_score" : null,
                    "hits" : [
                      {
                        "_index" : "cars",
                        "_type" : "_doc",
                        "_id" : "2",
                        "_score" : null,
                        "_source" : {
                          "vehicle_type" : "LCV",
                          "manufacturer_name" : "Ford",
                          "rental" : 599.99
                        },
                        "sort" : [
                          599.99
                        ]
                      }
                    ]
                  }
                }
              }
            ]
          }
        },
        {
          "key" : "02",
          "doc_count" : 2,
          "vehicle_type" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "CAR",
                "doc_count" : 2,
                "cheapest_vehicle" : {
                  "hits" : {
                    "total" : {
                      "value" : 2,
                      "relation" : "eq"
                    },
                    "max_score" : null,
                    "hits" : [
                      {
                        "_index" : "cars",
                        "_type" : "_doc",
                        "_id" : "4",
                        "_score" : null,
                        "_source" : {
                          "vehicle_type" : "CAR",
                          "manufacturer_name" : "Audi",
                          "rental" : 499.99
                        },
                        "sort" : [
                          499.99
                        ]
                      }
                    ]
                  }
                }
              }
            ]
          }
        },
        {
          "key" : "03",
          "doc_count" : 1,
          "vehicle_type" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "CAR",
                "doc_count" : 1,
                "cheapest_vehicle" : {
                  "hits" : {
                    "total" : {
                      "value" : 1,
                      "relation" : "eq"
                    },
                    "max_score" : null,
                    "hits" : [
                      {
                        "_index" : "cars",
                        "_type" : "_doc",
                        "_id" : "5",
                        "_score" : null,
                        "_source" : {
                          "vehicle_type" : "CAR",
                          "manufacturer_name" : "Audi",
                          "rental" : 399.99
                        },
                        "sort" : [
                          399.99
                        ]
                      }
                    ]
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

型

赞(0）回复(0）举报 5个月前

z3yyvxxp7#

aggs将工作在数字值默认情况下，如果你想工作的字符串字段，你应该启用它的文件

赞(0）回复(0）举报 5个月前