• Kubernetes Submit Queue's avatar
    Merge pull request #40727 from Random-Liu/handle-cri-in-place-upgrade · 4796c7b4
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue
    
    CRI: Handle cri in-place upgrade
    
    Fixes https://github.com/kubernetes/kubernetes/issues/40051.
    
    ## How does this PR restart/remove legacy containers/sandboxes?
    With this PR, dockershim will convert and return legacy containers and infra containers as regular containers/sandboxes. Then we can rely on the SyncPod logic to stop the legacy containers/sandboxes, and the garbage collector to remove the legacy containers/sandboxes.
    
    To forcibly trigger restart:
    * For infra containers, we manually set `hostNetwork` to opposite value to trigger a restart (See [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/kuberuntime_manager.go#L389))
    * For application containers, they will be restarted with the infra container.
    ## How does this PR avoid extra overhead when there is no legacy container/sandbox?
    For the lack of some labels, listing legacy containers needs extra `docker ps`. We should not introduce constant performance regression for legacy container cleanup. So we added the `legacyCleanupFlag`:
    * In `ListContainers` and `ListPodSandbox`, only do extra `ListLegacyContainers` and `ListLegacyPodSandbox` when `legacyCleanupFlag` is `NotDone`.
    * When dockershim starts, it will check whether there are legacy containers/sandboxes.
      * If there are none, it will mark `legacyCleanupFlag` as `Done`.
      * If there are any, it will leave `legacyCleanupFlag` as `NotDone`, and start a goroutine periodically check whether legacy cleanup is done.
    This makes sure that there is overhead only when there are legacy containers/sandboxes not cleaned up yet.
    
    ## Caveats
    * In-place upgrade will cause kubelet to restart all running containers.
    * RestartNever container will not be restarted.
    * Garbage collector sometimes keep the legacy containers for a long time if there aren't too many containers on the node. In that case, dockershim will keep performing extra `docker ps` which introduces overhead.
      * Manually remove all legacy containers will fix this.
      * Should we garbage collect legacy containers/sandboxes in dockershim by ourselves? /cc @yujuhong 
    * Host port will not be reclaimed for the lack of checkpoint for legacy sandboxes. https://github.com/kubernetes/kubernetes/pull/39903 /cc @freehan 
    
    /cc @yujuhong @feiskyer @dchen1107 @kubernetes/sig-node-api-reviews 
    **Release note**:
    
    ```release-note
    We should mention the caveats of in-place upgrade in release note.
    ```
    4796c7b4
Name
Last commit
Last update
..
api Loading commit data...
cadvisor Loading commit data...
client Loading commit data...
cm Loading commit data...
config Loading commit data...
container Loading commit data...
custommetrics Loading commit data...
dockershim Loading commit data...
dockertools Loading commit data...
envvars Loading commit data...
events Loading commit data...
eviction Loading commit data...
images Loading commit data...
kuberuntime Loading commit data...
leaky Loading commit data...
lifecycle Loading commit data...
metrics Loading commit data...
network Loading commit data...
pleg Loading commit data...
pod Loading commit data...
prober Loading commit data...
qos Loading commit data...
remote Loading commit data...
rkt Loading commit data...
rktshim Loading commit data...
secret Loading commit data...
server Loading commit data...
status Loading commit data...
sysctl Loading commit data...
types Loading commit data...
util Loading commit data...
volumemanager Loading commit data...
BUILD Loading commit data...
OWNERS Loading commit data...
active_deadline.go Loading commit data...
active_deadline_test.go Loading commit data...
disk_manager.go Loading commit data...
disk_manager_test.go Loading commit data...
doc.go Loading commit data...
kubelet.go Loading commit data...
kubelet_cadvisor.go Loading commit data...
kubelet_cadvisor_test.go Loading commit data...
kubelet_getters.go Loading commit data...
kubelet_getters_test.go Loading commit data...
kubelet_network.go Loading commit data...
kubelet_network_test.go Loading commit data...
kubelet_node_status.go Loading commit data...
kubelet_node_status_test.go Loading commit data...
kubelet_pods.go Loading commit data...
kubelet_pods_test.go Loading commit data...
kubelet_resources.go Loading commit data...
kubelet_resources_test.go Loading commit data...
kubelet_test.go Loading commit data...
kubelet_volumes.go Loading commit data...
kubelet_volumes_test.go Loading commit data...
networks.go Loading commit data...
oom_watcher.go Loading commit data...
oom_watcher_test.go Loading commit data...
pod_container_deletor.go Loading commit data...
pod_container_deletor_test.go Loading commit data...
pod_workers.go Loading commit data...
pod_workers_test.go Loading commit data...
reason_cache.go Loading commit data...
reason_cache_test.go Loading commit data...
runonce.go Loading commit data...
runonce_test.go Loading commit data...
runtime.go Loading commit data...
util.go Loading commit data...
volume_host.go Loading commit data...