kubernetes 基于标签的Prometheus过滤-错误

pvabu6sv  于 5个月前  发布在  Kubernetes
关注(0)|答案(2)|浏览(68)

我们使用这个查询来期望CPU指标的pod/namespace,并使用kube_pod_labels来获得基于标签的pod名称。

max(kube_pod_labels{label_workflow_instance_id="$workflow_instance_id", label_workflow_vertex_id=~"$workflow_vertex_id", pod=~".+"}  * on(pod) group_left sum by(pod) (label_replace(node_namespace_pod_container:container_cpu_usage_seconds_total:sum_irate{namespace="pipeline-worker", container_name!="POD"},"pod", "$1", "pod_name", "(.+)")) * on(pod) group_left kube_pod_status_phase{phase="Running"}) by(label_workflow_vertex_id)

字符串
能够看到空的查询结果。我们在EKS上有集群(附grafana UI截图)enter image description here
这里的任何帮助都是高度相关的(就像我在这里做错了什么)

holgip5t

holgip5t1#

将这些添加到k8s -telemetry-port=8081 -metric-labels-allowlist=pods=[*]中kube-state-metric deployment中的args中会对您有所帮助。
x1c 0d1x的数据

-------------------------------------

spec:
  containers:
  - args:
    - --port=8080
    - --telemetry-port=8081
    - --metric-labels-allowlist=pods=[*]
    - --resources=certificatesigningrequests,configmaps,cronjobs,daemonsets,deployments,endpoints,horizontalpodautoscalers,ingresses,jobs,leases,limitranges,mutatingwebhookconfigurations,namespaces,networkpolicies,nodes,persistentvolumeclaims,persistentvolumes,poddisruptionbudgets,pods,replicasets,replicationcontrollers,resourcequotas,secrets,services,statefulsets,storageclasses,validatingwebhookconfigurations,volumeattachments
    image: registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.9.2

字符串

8yparm6h

8yparm6h2#

www.asserts.ai中,我们使用以下记录规则记录容器CPU Metric
看起来您正在使用kube_mixin,因此mixin_pod_workload的记录规则可能已经存在于您的环境中

sum by (pod, container, namespace, node, job, workload, asserts_env, asserts_site) (
            rate(container_cpu_usage_seconds_total {container != "POD", image != "", container != ""}[5m])
            * on (pod, namespace, asserts_env, asserts_site) group_left(workload)
            mixin_pod_workload
            * on (pod, namespace, asserts_env, asserts_site) group_left()
            group by (pod, namespace, asserts_env, asserts_site) (kube_pod_labels)
          )

字符串

相关问题