• Kubernetes Submit Queue's avatar
    Merge pull request #37431 from liggitt/namespace-leftovers · b01e6f68
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue
    
    hold namespaces briefly before processing deletion
    
    possible fix for #36891
    
    in HA scenarios (either HA apiserver or HA etcd), it is possible for deletion of resources from namespace cleanup to race with creation of objects in the terminating namespace
    
    HA master timeline:
    1. "delete namespace n" API call goes to apiserver 1, deletion timestamp is set in etcd
    2. namespace controller observes namespace deletion, starts cleaning up resources, lists deployments
    3. "create deployment d" API call goes to apiserver 2, gets persisted to etcd
    4. apiserver 2 observes namespace deletion, stops allowing new objects to be created
    5. namespace controller finishes deleting listed deployments, deletes namespace
    
    HA etcd timeline:
    1. "create deployment d" API call goes to apiserver, gets persisted to etcd
    2. "delete namespace n" API call goes to apiserver, deletion timestamp is set in etcd
    3. namespace controller observes namespace deletion, starts cleaning up resources, lists deployments
    4. list call goes to non-leader etcd member that hasn't observed the new deployment or the deleted namespace yet
    5. namespace controller finishes deleting the listed deployments, deletes namespace
    
    In both cases, simply waiting to clean up the namespace (either for etcd members to observe objects created at the last second in the namespace, or for other apiservers to observe the namespace move to terminating phase and disallow additional creations) resolves the issue
    
    Possible other fixes:
    * do a second sweep of objects before deleting the namespace
    * have the namespace controller check for and clean up objects in namespaces that no longer exist
    * ...?
    b01e6f68
Name
Last commit
Last update
..
admission Loading commit data...
api Loading commit data...
apimachinery Loading commit data...
apis Loading commit data...
apiserver Loading commit data...
auth Loading commit data...
capabilities Loading commit data...
client Loading commit data...
cloudprovider Loading commit data...
controller Loading commit data...
conversion Loading commit data...
credentialprovider Loading commit data...
dns Loading commit data...
fieldpath Loading commit data...
fields Loading commit data...
generated Loading commit data...
genericapiserver Loading commit data...
healthz Loading commit data...
httplog Loading commit data...
hyperkube Loading commit data...
kubectl Loading commit data...
kubelet Loading commit data...
kubemark Loading commit data...
labels Loading commit data...
master Loading commit data...
metrics Loading commit data...
probe Loading commit data...
proxy Loading commit data...
quota Loading commit data...
registry Loading commit data...
routes Loading commit data...
runtime Loading commit data...
security Loading commit data...
securitycontext Loading commit data...
selection Loading commit data...
serviceaccount Loading commit data...
ssh Loading commit data...
storage Loading commit data...
types Loading commit data...
util Loading commit data...
version Loading commit data...
volume Loading commit data...
watch Loading commit data...
OWNERS Loading commit data...