Skip to content

K
k3s
Unverified Commit 8ae6202b authored by Kubernetes Submit Queue's avatar Kubernetes Submit Queue Committed by GitHub
Browse files
Options

Merge pull request #56871 from jpbetz/etcd-version-monitor-3.1

Automatic merge from submit-queue (batch tested with PRs 56650, 55813, 56911, 56921, 56871). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Improve etcd-version-monitor metrics proxying, add etcd 3.1 gprc metr… Partially addresses https://github.com/kubernetes/kubernetes/issues/56869: - Fix `etcd-version-monitor` to support etcd 3.1: Add support for the etcd 3.1+ [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus) metrics format, which backward incompatibly replaces the 3.0 legacy grpc metric format. Expose the go-grpc-prometheus metrics both in the 3.1 format and in the 3.0 format so systems consuming `etcd-version-monitor` metrics have a clean, simple upgrade path. - Expose *all* etcd metrics by default, making this a one stop shop for all etcd metrics. - Expose grpc request latency histogram metrics (`grpc_server_handling_seconds` from [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus) metrics format). Rewrite etcd 3.0 legacy metric for latency histograms to the etcd 3.1+ `go-grpc-prometheus` format so there is a single format exported for all etcd versions. etcd 3.0 to 3.1 upgrade path: Continue to use the `etcd_grpc_requests_total`. Once the upgrade is complete and all etcd nodes are running 3.1, migrate to the `grpc_server_handled_total` metric at your leisure. This PR reorganizes the code substantially. Previously, the code to proxy etcd metrics was hard coded and limited to a single counter metric. This has been entirely replaced with code that generically filters, rewrites proxied etcd metrics and then aggregates them with custom metrics such as the etcd version metric. cc @wojtek-t @mml @shyamjvs @cheftako ```release-note Fix `etcd-version-monitor` to backward compatibly support etcd 3.1 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus) metrics format. ```
parents ea55d70e a0874620
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 186 additions and 69 deletions
cluster/images/etcd-version-monitor/BUILD
View file @ 8ae6202b
...@@ -19,6 +19,8 @@ go_library( ...@@ -19,6 +19,8 @@ go_library(
deps = [ deps = [
"//vendor/github.com/golang/glog:go_default_library", "//vendor/github.com/golang/glog:go_default_library",
"//vendor/github.com/prometheus/client_golang/prometheus:go_default_library", "//vendor/github.com/prometheus/client_golang/prometheus:go_default_library",
"//vendor/github.com/prometheus/client_golang/prometheus/promhttp:go_default_library",
"//vendor/github.com/prometheus/client_model/go:go_default_library",
"//vendor/github.com/prometheus/common/expfmt:go_default_library", "//vendor/github.com/prometheus/common/expfmt:go_default_library",
"//vendor/github.com/spf13/pflag:go_default_library", "//vendor/github.com/spf13/pflag:go_default_library",
], ],
......
cluster/images/etcd-version-monitor/Makefile
View file @ 8ae6202b
...@@ -20,7 +20,7 @@ ...@@ -20,7 +20,7 @@
ARCH:=amd64 ARCH:=amd64
GOLANG_VERSION?=1.8.3 GOLANG_VERSION?=1.8.3
REGISTRY?=gcr.io/google-containers REGISTRY?=gcr.io/google-containers
TAG?=0.1.0 TAG?=0.1.1
IMAGE:=$(REGISTRY)/etcd-version-monitor:$(TAG) IMAGE:=$(REGISTRY)/etcd-version-monitor:$(TAG)
CURRENT_DIR:=$(pwd) CURRENT_DIR:=$(pwd)
TEMP_DIR:=$(shell mktemp -d) TEMP_DIR:=$(shell mktemp -d)
......
cluster/images/etcd-version-monitor/README.md
View file @ 8ae6202b
# etcd-version-monitor # etcd-version-monitor
This is a tool for exporting metrics related to etcd version, like etcd This is a tool for exporting etcd metrics and supplementing them with etcd
server's binary version, cluster version, and counts of different kinds of server binary version and cluster version. These metrics are in
gRPC calls (which is a characteristic of v3), etc. These metrics are in
prometheus format and can be scraped by a prometheus server. prometheus format and can be scraped by a prometheus server.
The metrics are exposed at the http://localhost:9101/metrics endpoint. The metrics are exposed at the http://localhost:9101/metrics endpoint.
For etcd 3.1+, the
[go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus)
metrics format, which backward incompatibly replaces the 3.0 legacy grpc metric
format, is exposed in both the 3.1 format and in the 3.0. This preserves
backward compatiblity.
For etcd 3.1+, the `--metrics=extensive` must be set on etcd for grpc request
latency metrics (`etcd_grpc_unary_requests_duration_seconds`) to be exposed.
**RUNNING THE TOOL** **RUNNING THE TOOL**
To run this tool as a docker container: To run this tool as a docker container:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment