• Kubernetes Submit Queue's avatar
    Merge pull request #53994 from vmware/e2e-ClaimDeleteBeforePod-diff · 21490ec5
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    Updating E2E test for deleting PVC when PVC is in use
    
    **What this PR does / why we need it**:
    This test updates an existing e2e test and adds extra verification.
    Updated workflow of the test is as below
    1. Create PVC, Wait until PV is provisioned. Create POD using PVC.
    2. Verify POD is running and PV is attached to the node.
    3. Delete PVC.
    4. Verify Volume remains attached to the pod after deleting claim.
    5. Verify Volume is accessible in the pod after deleting claim.
    6. Verify associated PV is present and its status should be failed.
    7. Delete Pod and wait until PV is unmounted and detached from the Node.
    6. Wait and Verify PV is deleted after POD is deleted.
    
    
    
    **Which issue this PR fixes**
    fixes # https://github.com/vmware/kubernetes/issues/279
    
    **Special notes for your reviewer**:
    Test logs
    ```
    # go run hack/e2e.go --check-version-skew=false -v -test --test_args='--ginkgo.focus=should\snot\sdetach\sand\sunmount\sPV\swhen\sassociated\spvc\swith\sdelete\sas\sreclaimPolicy\sis\sdeleted\swhen\sit\sis\sin\suse\sby\sthe\spod'
    flag provided but not defined: -check-version-skew
    Usage of /tmp/go-build371606839/command-line-arguments/_obj/exe/e2e:
      -get
        	go get -u kubetest if old or not installed (default true)
      -old duration
        	Consider kubetest old if it exceeds this (default 24h0m0s)
    2017/10/16 15:42:40 e2e.go:55: NOTICE: go run hack/e2e.go is now a shim for test-infra/kubetest
    2017/10/16 15:42:40 e2e.go:56:   Usage: go run hack/e2e.go [--get=true] [--old=24h0m0s] -- [KUBETEST_ARGS]
    2017/10/16 15:42:40 e2e.go:57:   The separator is required to use --get or --old flags
    2017/10/16 15:42:40 e2e.go:58:   The -- flag separator also suppresses this message
    2017/10/16 15:42:40 e2e.go:77: Calling kubetest --check-version-skew=false -v -test --test_args=--ginkgo.focus=should\snot\sdetach\sand\sunmount\sPV\swhen\sassociated\spvc\swith\sdelete\sas\sreclaimPolicy\sis\sdeleted\swhen\sit\sis\sin\suse\sby\sthe\spod...
    2017/10/16 15:42:40 util.go:154: Running: ./cluster/kubectl.sh --match-server-version=false version
    2017/10/16 15:42:40 util.go:156: Step './cluster/kubectl.sh --match-server-version=false version' finished in 293.775296ms
    2017/10/16 15:42:40 util.go:154: Running: ./hack/e2e-internal/e2e-status.sh
    Skeleton Provider: prepare-e2e not implemented
    Client Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-alpha.1.913+297ab03890a6a7-dirty", GitCommit:"297ab03890a6a76f268eb5415e0fb16f20b2309e", GitTreeState:"dirty", BuildDate:"2017-10-16T20:50:38Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
    Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-alpha.1.1181+77b83e44", GitCommit:"77b83e44", GitTreeState:"clean", BuildDate:"2017-10-16T07:07:02Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
    2017/10/16 15:42:40 util.go:156: Step './hack/e2e-internal/e2e-status.sh' finished in 317.940582ms
    2017/10/16 15:42:40 util.go:154: Running: ./hack/ginkgo-e2e.sh --ginkgo.focus=should\snot\sdetach\sand\sunmount\sPV\swhen\sassociated\spvc\swith\sdelete\sas\sreclaimPolicy\sis\sdeleted\swhen\sit\sis\sin\suse\sby\sthe\spod
    Conformance test: not doing test setup.
    Oct 16 15:42:42.327: INFO: Overriding default scale value of zero to 1
    Oct 16 15:42:42.327: INFO: Overriding default milliseconds value of zero to 5000
    I1016 15:42:42.577720    8325 e2e.go:369] Starting e2e run "51f11717-b2c3-11e7-bd54-0050569c26b8" on Ginkgo node 1
    Running Suite: Kubernetes e2e suite
    ===================================
    Random Seed: 1508193761 - Will randomize all specs
    Will run 1 of 706 specs
    
    Oct 16 15:42:42.678: INFO: >>> kubeConfig: /root/.kube/config
    Oct 16 15:42:42.686: INFO: Waiting up to 4h0m0s for all (but 0) nodes to be schedulable
    Oct 16 15:42:42.724: INFO: Waiting up to 10m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready
    Oct 16 15:42:42.883: INFO: 13 / 13 pods in namespace 'kube-system' are running and ready (0 seconds elapsed)
    Oct 16 15:42:42.883: INFO: expected 4 pod replicas in namespace 'kube-system', 4 are Running and Ready.
    Oct 16 15:42:42.891: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller]
    Oct 16 15:42:42.891: INFO: Dumping network health container logs from all nodes...
    SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
    ------------------------------
    [sig-storage] PersistentVolumes [Feature:ReclaimPolicy] [sig-storage] persistentvolumereclaim:vsphere 
      should not detach and unmount PV when associated pvc with delete as reclaimPolicy is deleted when it is in use by the pod
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/pv_reclaimpolicy.go:136
    [BeforeEach] [sig-storage] PersistentVolumes [Feature:ReclaimPolicy]
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:133
    STEP: Creating a kubernetes client
    Oct 16 15:42:42.904: INFO: >>> kubeConfig: /root/.kube/config
    STEP: Building a namespace api object
    STEP: Waiting for a default service account to be provisioned in namespace
    [BeforeEach] [sig-storage] PersistentVolumes [Feature:ReclaimPolicy]
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/pv_reclaimpolicy.go:48
    Oct 16 15:42:42.994: INFO: Waiting up to 4h0m0s for all (but 0) nodes to be schedulable
    [BeforeEach] [sig-storage] persistentvolumereclaim:vsphere
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/pv_reclaimpolicy.go:56
    [It] should not detach and unmount PV when associated pvc with delete as reclaimPolicy is deleted when it is in use by the pod
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/pv_reclaimpolicy.go:136
    STEP: running testSetupVSpherePersistentVolumeReclaim
    STEP: creating vmdk
    STEP: creating the pv
    STEP: creating the pvc
    Oct 16 15:42:44.595: INFO: Waiting for PV vspherepv-ksccp to bind to PVC pvc-n4rq7
    Oct 16 15:42:44.595: INFO: Waiting up to 5m0s for PersistentVolumeClaim pvc-n4rq7 to have phase Bound
    Oct 16 15:42:44.606: INFO: PersistentVolumeClaim pvc-n4rq7 found but phase is Pending instead of Bound.
    Oct 16 15:42:47.625: INFO: PersistentVolumeClaim pvc-n4rq7 found and phase=Bound (3.029926391s)
    Oct 16 15:42:47.625: INFO: Waiting up to 5m0s for PersistentVolume vspherepv-ksccp to have phase Bound
    Oct 16 15:42:47.632: INFO: PersistentVolume vspherepv-ksccp found and phase=Bound (6.598243ms)
    STEP: Creating the Pod
    STEP: Deleting the Claim
    Oct 16 15:42:59.709: INFO: Deleting PersistentVolumeClaim "pvc-n4rq7"
    STEP: Verify the volume is attached to the node
    STEP: Verify the volume is accessible and available in the pod
    Oct 16 15:43:00.076: INFO: Running '/root/divyenp/kubernetes/_output/bin/kubectl --server=https://10.192.55.64 --kubeconfig=/root/.kube/config exec pvc-tester-r9ww9 --namespace=e2e-tests-persistentvolumereclaim-6pfpf -- /bin/touch /mnt/volume1/emptyFile.txt'
    Oct 16 15:43:00.604: INFO: stderr: ""
    Oct 16 15:43:00.604: INFO: stdout: ""
    Oct 16 15:43:00.604: INFO: Verified that Volume is accessible in the POD after deleting PV claim
    Oct 16 15:43:00.610: INFO: Waiting up to 1m0s for PersistentVolume vspherepv-ksccp to have phase Failed
    Oct 16 15:43:00.619: INFO: PersistentVolume vspherepv-ksccp found and phase=Failed (9.016306ms)
    STEP: Deleting the Pod
    Oct 16 15:43:00.619: INFO: Deleting pod pvc-tester-r9ww9
    Oct 16 15:43:00.650: INFO: Waiting up to 5m0s for pod "pvc-tester-r9ww9" in namespace "e2e-tests-persistentvolumereclaim-6pfpf" to be "terminated due to deadline exceeded"
    Oct 16 15:43:00.668: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 18.507993ms
    Oct 16 15:43:02.675: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 2.024854663s
    Oct 16 15:43:04.682: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 4.03197856s
    Oct 16 15:43:06.688: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 6.037718623s
    Oct 16 15:43:08.697: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 8.047192574s
    Oct 16 15:43:10.703: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 10.052754761s
    Oct 16 15:43:12.708: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 12.057876018s
    Oct 16 15:43:14.714: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 14.063962712s
    Oct 16 15:43:16.719: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 16.068826626s
    Oct 16 15:43:18.725: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 18.074735397s
    Oct 16 15:43:20.730: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 20.080498293s
    Oct 16 15:43:22.736: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 22.086586123s
    Oct 16 15:43:24.742: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 24.092219324s
    Oct 16 15:43:26.747: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 26.097385301s
    Oct 16 15:43:28.753: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 28.103127591s
    Oct 16 15:43:30.758: INFO: Pod "pvc-tester-r9ww9": Phase="Running", Reason="", readiness=true. Elapsed: 30.108014823s
    Oct 16 15:43:32.764: INFO: Pod "pvc-tester-r9ww9": Phase="Pending", Reason="", readiness=false. Elapsed: 32.113847674s
    Oct 16 15:43:34.772: INFO: Pod "pvc-tester-r9ww9": Phase="Pending", Reason="", readiness=false. Elapsed: 34.122010171s
    Oct 16 15:43:36.787: INFO: Pod "pvc-tester-r9ww9" in namespace "e2e-tests-persistentvolumereclaim-6pfpf" not found. Error: pods "pvc-tester-r9ww9" not found
    Oct 16 15:43:36.787: INFO: Ignore "not found" error above. Pod "pvc-tester-r9ww9" successfully deleted
    STEP: Verify PV is detached from the node after Pod is deleted
    Oct 16 15:43:46.913: INFO: Waiting for Volume "[vsanDatastore] 1874c359-f300-a0cc-fd7e-02002a623c85/e2e-vmdk-1508193763110460154.vmdk" to detach from "kubernetes-node2".
    Oct 16 15:43:56.918: INFO: Waiting for Volume "[vsanDatastore] 1874c359-f300-a0cc-fd7e-02002a623c85/e2e-vmdk-1508193763110460154.vmdk" to detach from "kubernetes-node2".
    Oct 16 15:44:06.905: INFO: Volume "[vsanDatastore] 1874c359-f300-a0cc-fd7e-02002a623c85/e2e-vmdk-1508193763110460154.vmdk" appears to have successfully detached from "kubernetes-node2".
    STEP: Verify PV should be deleted automatically
    Oct 16 15:44:06.905: INFO: Waiting up to 30s for PersistentVolume vspherepv-ksccp to get deleted
    Oct 16 15:44:06.909: INFO: PersistentVolume vspherepv-ksccp was removed
    [AfterEach] [sig-storage] persistentvolumereclaim:vsphere
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/pv_reclaimpolicy.go:62
    STEP: running testCleanupVSpherePersistentVolumeReclaim
    Oct 16 15:44:06.962: INFO: Deleting PersistentVolume "vspherepv-ksccp"
    [AfterEach] [sig-storage] PersistentVolumes [Feature:ReclaimPolicy]
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:134
    Oct 16 15:44:06.967: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
    STEP: Destroying namespace "e2e-tests-persistentvolumereclaim-6pfpf" for this suite.
    Oct 16 15:44:15.325: INFO: namespace: e2e-tests-persistentvolumereclaim-6pfpf, resource: bindings, ignored listing per whitelist
    Oct 16 15:44:15.638: INFO: namespace e2e-tests-persistentvolumereclaim-6pfpf deletion completed in 8.651759385s
    
    • [SLOW TEST:92.734 seconds]
    [sig-storage] PersistentVolumes [Feature:ReclaimPolicy]
    /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/framework.go:22
      [sig-storage] persistentvolumereclaim:vsphere
      /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/framework.go:22
        should not detach and unmount PV when associated pvc with delete as reclaimPolicy is deleted when it is in use by the pod
        /root/divyenp/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/storage/pv_reclaimpolicy.go:136
    ------------------------------
    SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSOct 16 15:44:15.651: INFO: Running AfterSuite actions on all node
    Oct 16 15:44:15.651: INFO: Running AfterSuite actions on node 1
    
    Ran 1 of 706 Specs in 92.974 seconds
    SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 705 Skipped PASS
    
    Ginkgo ran 1 suite in 1m33.830856163s
    Test Suite Passed
    2017/10/16 15:44:15 util.go:156: Step './hack/ginkgo-e2e.sh --ginkgo.focus=should\snot\sdetach\sand\sunmount\sPV\swhen\sassociated\spvc\swith\sdelete\sas\sreclaimPolicy\sis\sdeleted\swhen\sit\sis\sin\suse\sby\sthe\spod' finished in 1m34.75838192s
    2017/10/16 15:44:15 e2e.go:81: Done
    ```
    VVMware Reviewers: @rohitjogvmw @BaluDontu @tusharnt
    **Release note**:
    
    ```release-note
    None
    ```
    21490ec5
Name
Last commit
Last update
..
apimachinery Loading commit data...
apps Loading commit data...
auth Loading commit data...
autoscaling Loading commit data...
chaosmonkey Loading commit data...
common Loading commit data...
framework Loading commit data...
generated Loading commit data...
instrumentation Loading commit data...
kubectl Loading commit data...
lifecycle Loading commit data...
manifest Loading commit data...
multicluster Loading commit data...
network Loading commit data...
node Loading commit data...
perftype Loading commit data...
scalability Loading commit data...
scheduling Loading commit data...
servicecatalog Loading commit data...
storage Loading commit data...
testing-manifests Loading commit data...
ui Loading commit data...
upgrades Loading commit data...
BUILD Loading commit data...
README.md Loading commit data...
e2e-example-config.json Loading commit data...
e2e.go Loading commit data...
e2e_test.go Loading commit data...
events.go Loading commit data...
example_cluster_dns.go Loading commit data...
examples.go Loading commit data...
gke_local_ssd.go Loading commit data...
gke_node_pools.go Loading commit data...
network_partition.go Loading commit data...
pod_gc.go Loading commit data...
pods.go Loading commit data...
pre_stop.go Loading commit data...
ssh.go Loading commit data...