• Kubernetes Submit Queue's avatar
    Merge pull request #63887 from luxas/kubeadm_kubelet_integration · 773ced14
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue (batch tested with PRs 63914, 63887, 64116, 64026, 62933). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    kubeadm: Write kubelet config file to disk and persist in-cluster
    
    **What this PR does / why we need it**:
    In order to make configuration flow from the cluster level to node level, we need a way for kubeadm to tell the kubelet what config to use. As of v1.10 (I think) the kubelet can read `--config` using the kubelet Beta ComponentConfiguration API, so now we have an interface to talk to the kubelet properly.
    
    This PR:
     - Writes the kubelet ComponentConfig to `/var/lib/kubelet/config.yaml` on init and join
     - Writes an environment file to source in the kubelet systemd dropin `/var/lib/kubelet/kubeadm-flags.env`. This file contain runtime flags that should be passed to the kubelet.
     - Uploads a ConfigMap with the name `kubelet-config-1.X`
     - Patches the node object so that it starts using the ConfigMap with updates using Dynamic Kubelet Configuration, **only if the feature gate is set** (currently alpha and off by default, not intended to be switched on in v1.11)
     - Updates the phase commands to reflect this new flow
    
    The kubelet dropin file I used now looks like this:
    ```
    # v1.11.x dropin as-is at HEAD
    # /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
    ---
    [Service]
    Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
    Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
    EnvironmentFile-=/var/lib/kubelet/kubeadm-flags.env
    # Should default to 0 in v1.11: https://github.com/kubernetes/kubernetes/pull/63881, and hence not be here in the real v1.11 manifest
    Environment="KUBELET_CADVISOR_ARGS=--cadvisor-port=0"
    # Should be configurable via the config file: https://github.com/kubernetes/kubernetes/issues/63878, and hence be configured using the file in v1.11
    Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true"
    ExecStart=
    ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS
    ---
    # v1.11.x dropin end goal
    # /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
    ---
    [Service]
    Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
    Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
    EnvironmentFile-=/var/lib/kubelet/kubeadm-flags.env
    ExecStart=
    ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
    ---
    # Environment file dynamically created at runtime by "kubeadm init"
    # /var/lib/kubelet/kubeadm-flags.env
    KUBELET_KUBEADM_ARGS=--cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni
    ```
    
    **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
    Fixes https://github.com/kubernetes/kubeadm/issues/822
    Fixes https://github.com/kubernetes/kubeadm/issues/571
    
    **Special notes for your reviewer**:
    
    **Release note**:
    
    ```release-note
    "kubeadm init" now writes a structured and versioned kubelet ComponentConfiguration file to `/var/lib/kubelet/config.yaml` and an environment file with runtime flags (you can source this file in the systemd kubelet dropin) to `/var/lib/kubelet/kubeadm-flags.env`.
    ```
    @kubernetes/sig-cluster-lifecycle-pr-reviews @mtaufen
    773ced14
Name
Last commit
Last update
.github Loading commit data...
Godeps Loading commit data...
api Loading commit data...
build Loading commit data...
cluster Loading commit data...
cmd Loading commit data...
docs Loading commit data...
hack Loading commit data...
logo Loading commit data...
pkg Loading commit data...
plugin Loading commit data...
staging Loading commit data...
test Loading commit data...
third_party Loading commit data...
translations Loading commit data...
vendor Loading commit data...
.bazelrc Loading commit data...
.generated_files Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.kazelcfg.json Loading commit data...
BUILD.bazel Loading commit data...
CHANGELOG-1.10.md Loading commit data...
CHANGELOG-1.11.md Loading commit data...
CHANGELOG-1.2.md Loading commit data...
CHANGELOG-1.3.md Loading commit data...
CHANGELOG-1.4.md Loading commit data...
CHANGELOG-1.5.md Loading commit data...
CHANGELOG-1.6.md Loading commit data...
CHANGELOG-1.7.md Loading commit data...
CHANGELOG-1.8.md Loading commit data...
CHANGELOG-1.9.md Loading commit data...
CHANGELOG.md Loading commit data...
CONTRIBUTING.md Loading commit data...
LICENSE Loading commit data...
Makefile Loading commit data...
Makefile.generated_files Loading commit data...
OWNERS Loading commit data...
OWNERS_ALIASES Loading commit data...
README.md Loading commit data...
SUPPORT.md Loading commit data...
WORKSPACE Loading commit data...
code-of-conduct.md Loading commit data...
labels.yaml Loading commit data...