• Kubernetes Submit Queue's avatar
    Merge pull request #37431 from liggitt/namespace-leftovers · b01e6f68
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue
    
    hold namespaces briefly before processing deletion
    
    possible fix for #36891
    
    in HA scenarios (either HA apiserver or HA etcd), it is possible for deletion of resources from namespace cleanup to race with creation of objects in the terminating namespace
    
    HA master timeline:
    1. "delete namespace n" API call goes to apiserver 1, deletion timestamp is set in etcd
    2. namespace controller observes namespace deletion, starts cleaning up resources, lists deployments
    3. "create deployment d" API call goes to apiserver 2, gets persisted to etcd
    4. apiserver 2 observes namespace deletion, stops allowing new objects to be created
    5. namespace controller finishes deleting listed deployments, deletes namespace
    
    HA etcd timeline:
    1. "create deployment d" API call goes to apiserver, gets persisted to etcd
    2. "delete namespace n" API call goes to apiserver, deletion timestamp is set in etcd
    3. namespace controller observes namespace deletion, starts cleaning up resources, lists deployments
    4. list call goes to non-leader etcd member that hasn't observed the new deployment or the deleted namespace yet
    5. namespace controller finishes deleting the listed deployments, deletes namespace
    
    In both cases, simply waiting to clean up the namespace (either for etcd members to observe objects created at the last second in the namespace, or for other apiservers to observe the namespace move to terminating phase and disallow additional creations) resolves the issue
    
    Possible other fixes:
    * do a second sweep of objects before deleting the namespace
    * have the namespace controller check for and clean up objects in namespaces that no longer exist
    * ...?
    b01e6f68
Name
Last commit
Last update
..
certificates Loading commit data...
cronjob Loading commit data...
daemon Loading commit data...
deployment Loading commit data...
disruption Loading commit data...
endpoint Loading commit data...
garbagecollector Loading commit data...
informers Loading commit data...
job Loading commit data...
namespace Loading commit data...
node Loading commit data...
petset Loading commit data...
podautoscaler Loading commit data...
podgc Loading commit data...
replicaset Loading commit data...
replication Loading commit data...
resourcequota Loading commit data...
route Loading commit data...
service Loading commit data...
serviceaccount Loading commit data...
volume Loading commit data...
.import-restrictions Loading commit data...
BUILD Loading commit data...
OWNERS Loading commit data...
client_builder.go Loading commit data...
controller_ref_manager.go Loading commit data...
controller_utils.go Loading commit data...
controller_utils_test.go Loading commit data...
doc.go Loading commit data...
lookup_cache.go Loading commit data...