• Kubernetes Submit Queue's avatar
    Merge pull request #65594 from liggitt/node-csr-addresses-2 · 337dfe0a
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue (batch tested with PRs 65052, 65594). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    Derive kubelet serving certificate CSR template from node status addresses
    
    xref https://github.com/kubernetes/features/issues/267
    fixes #55633
    
    Builds on https://github.com/kubernetes/kubernetes/pull/65587
    
    * Makes the cloud provider authoritative when recording node status addresses
    * Makes the node status addresses authoritative for the kube-apiserver determining how to speak to a kubelet (stops paying attention to the hostname label when determining how to reach a kubelet, which was only done to support kubelets < 1.5)
    * Updates kubelet certificate rotation to be driven from node status
      * Avoids needing to compute node addresses a second time, and differently, in order to request serving certificates.
      * Allows the kubelet to react to changes in its status addresses by updating its serving certificate
      * Allows the kubelet to be driven by external cloud providers recording node addresses on the node status
    
    test procedure:
    ```sh
    # setup
    export FEATURE_GATES=RotateKubeletServerCertificate=true
    export KUBELET_FLAGS="--rotate-server-certificates=true --cloud-provider=external"
    
    # cleanup from previous runs
    sudo rm -fr /var/lib/kubelet/pki/
    
    # startup
    hack/local-up-cluster.sh
    
    # wait for a node to register, verify it didn't set addresses
    kubectl get nodes 
    kubectl get node/127.0.0.1 -o jsonpath={.status.addresses}
    
    # verify the kubelet server isn't available, and that it didn't populate a serving certificate
    curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
    ls -la /var/lib/kubelet/pki
    
    # set an address on the node
    curl -X PATCH http://localhost:8080/api/v1/nodes/127.0.0.1/status \
      -H "Content-Type: application/merge-patch+json" \
      --data '{"status":{"addresses":[{"type":"Hostname","address":"localhost"}]}}'
    
    # verify a csr was submitted with the right SAN, and approve it
    kubectl describe csr
    kubectl certificate approve csr-...
    
    # verify the kubelet connection uses a cert that is properly signed and valid for the specified hostname, but NOT the IP
    curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
    curl --cacert _output/certs/server-ca.crt -v https://127.0.0.1:10250/pods
    ls -la /var/lib/kubelet/pki
    
    # set an hostname and IP address on the node
    curl -X PATCH http://localhost:8080/api/v1/nodes/127.0.0.1/status \
      -H "Content-Type: application/merge-patch+json" \
      --data '{"status":{"addresses":[{"type":"Hostname","address":"localhost"},{"type":"InternalIP","address":"127.0.0.1"}]}}'
    
    # verify a csr was submitted with the right SAN, and approve it
    kubectl describe csr
    kubectl certificate approve csr-...
    
    # verify the kubelet connection uses a cert that is properly signed and valid for the specified hostname AND IP
    curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
    curl --cacert _output/certs/server-ca.crt -v https://127.0.0.1:10250/pods
    ls -la /var/lib/kubelet/pki
    ```
    
    ```release-note
    * kubelets that specify `--cloud-provider` now only report addresses in Node status as determined by the cloud provider
    * kubelet serving certificate rotation now reacts to changes in reported node addresses, and will request certificates for addresses set by an external cloud provider
    ```
    337dfe0a
Name
Last commit
Last update
..
async Loading commit data...
bandwidth Loading commit data...
config Loading commit data...
configz Loading commit data...
conntrack Loading commit data...
dbus Loading commit data...
ebtables Loading commit data...
env Loading commit data...
file Loading commit data...
filesystem Loading commit data...
flag Loading commit data...
flock Loading commit data...
goroutinemap Loading commit data...
hash Loading commit data...
initsystem Loading commit data...
interrupt Loading commit data...
io Loading commit data...
ipconfig Loading commit data...
ipset Loading commit data...
iptables Loading commit data...
ipvs Loading commit data...
keymutex Loading commit data...
labels Loading commit data...
limitwriter Loading commit data...
maps Loading commit data...
metrics Loading commit data...
mount Loading commit data...
net Loading commit data...
netsh Loading commit data...
node Loading commit data...
normalizer Loading commit data...
nsenter Loading commit data...
oom Loading commit data...
parsers Loading commit data...
pod Loading commit data...
pointer Loading commit data...
procfs Loading commit data...
reflector/prometheus Loading commit data...
removeall Loading commit data...
resizefs Loading commit data...
resourcecontainer Loading commit data...
rlimit Loading commit data...
selinux Loading commit data...
slice Loading commit data...
strings Loading commit data...
sysctl Loading commit data...
system Loading commit data...
tail Loading commit data...
taints Loading commit data...
template Loading commit data...
term Loading commit data...
threading Loading commit data...
tolerations Loading commit data...
version Loading commit data...
workqueue/prometheus Loading commit data...
BUILD Loading commit data...
verify-util-pkg.sh Loading commit data...