Automatic merge from submit-queue

AWS kube-up: Remove SecurityContextDeny admission controller (to mirror GCE)

This PR allows the user to tune the ADMISSION_CONTROL options for AWS environments (much like local-up-cluser.sh). The main impetus is to allow users to exclude the SecurityContextDeny admission controller which causes e2e AWS based tests to fail with `pod.Spec.SecurityContext.SELinuxOptions is forbidden`.

Now AWS e2e tests are happy and can actually go green.

Automatic merge from submit-queue

Add ListOptions to RBAC group

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Otherwise clientset complains about unregistered kind when ListOptions
are passed.

Automatic merge from submit-queue

Fix third party

Fixes https://github.com/kubernetes/kubernetes/issues/25421
Fixes https://github.com/kubernetes/kubernetes/issues/25422

@AdoHe @sjenning @caesarxuchao @lavalamp 

@kubernetes/sig-api-machinery 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Node e2e use vendored testing packages.


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

update ingress describe output

refer #26684 for more detail. @bprashanth ptal.

Otherwise clientset complains about unregistered kind when options
are passed.

Automatic merge from submit-queue

Re-enable node problem detector by default

Re-enable node problem detector started in gce cluster by default.

For now, in the master node, the node problem detector will be started and do nothing (see https://github.com/kubernetes/node-problem-detector/pull/13).

But in fact, in my test cluster, the master has no extra cpu to run the node problem detector, so node problem detector is started on all nodes except master, which is what we want but not expected...

@dchen1107 
/cc @kubernetes/sig-node 
/cc @andyzheng0831 for the gci script change.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Fix node e2e image pulling retry logic

Automatic merge from submit-queue

Fixing logic to generate ExternalHost in genericapiserver

@ncdc pointed it out (https://kubernetes.slack.com/archives/sig-api-machinery/p1464974528000139) that lines 305 and 306 dont match. We should be using ReadWritePort instead of ServiceReadWritePort in line 306.

https://github.com/kubernetes/kubernetes/pull/20626 seems to be the culprit PR.

Automatic merge from submit-queue

Expose printer utils that append labels

So that it can be reused downstream. @smarterclayton 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Don't run fluentd-es on GCI masters

It isn't run on containervm masters. It can't do anything on the master because the master doesn't have kube-proxy running to enable fluentd to talk to the elasticsearch service.

@andyzheng0831 

Automatic merge from submit-queue

Add collection of the new glbc and cluster-autoscaler logs

I've incremented the version numbers by 2 to avoid conflicting with #26652. I'll make sure the potential conflict between the images gets resolved reasonably.

cc @piosz @bprashanth @aledbf 

Automatic merge from submit-queue

Set Dashboard UI version to v1.1.0-beta2

This is our second beta. Next will come weekly till we reach final v1.1 version.

https://github.com/kubernetes/dashboard/releases/tag/v1.1.0-beta2

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Improved check in shouldn't increase cluster size e2e test.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Improved check in shouldn't increase cluster size e2e test.

Automatic merge from submit-queue

rkt: Wrap exec errors as utilexec.ExitError

This is needed by the exec prober to distinguish error types and exit
codes correctly. Without this, the exec prober used for liveness probes
doesn't identify errors correctly and restarts aren't triggered. Fixes #26456

An alternative, and preferable solution would be to use utilexec
everywhere, but that change is much more involved and should come at a
later date. Unfortunately, until that change is made, writing tests for
this is quite difficult.

cc @yifan-gu @sjpotter 

Automatic merge from submit-queue

Fix the problem in building hyperkube for 3rd party registry

Minor bugfix.  fixes #26620



[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Update Node e2e Core OS image to run systemd with CPU & Memory accounting enabled by default

cc @derekwaynecarr 

For #26289

Automatic merge from submit-queue

Update kubectl describe pod to use pod level qos

Fixes https://github.com/kubernetes/kubernetes/issues/26374

/cc @vishh 

Automatic merge from submit-queue

Fixes 25890 flake. Let GC convert ListOptions to v1 before passing it to the dynamic client

GC's ListWatcher directly passed the api.ListOptions to the dynamic client, but the parameter codec of dynamic client converts the options to queries based on the tags in the struct, which are not present in api.ListOptions, so the queries are not sent to the server. As a result, the Watch request was sent without a resourceVersion, causing missed events. Flake #25890 is caused by the missed deletion events.

This PR converts the api.ListOptions to v1.ListOptions before the GC passes it to the dynamic codec. The flaky test has successfully passed 79 times ([log](https://00e9e64bacd064560a027fbee9c5a373a1614f3a56e652ae40-apidata.googleusercontent.com/download/storage/v1_internal/b/kubernetes-jenkins/o/pr-logs%2Fpull%2F25923%2Fkubernetes-pull-test-unit-integration%2F28364%2Fbuild-log.txt?qk=AD5uMEv72OjSUqDyk5i-ZLurcmM4i7gket1c7WaqR7yuIYz7WhPYT7ewVBafijV0ymnPTYqxRYt1kp6S9YQv7chPwC-3UtrKetKfhYnvAFrPGXAIBxHytTmpFohRAYgsARN1B6j1f9vyK5lM-8jyzRGhCK3sCRsAPnbDBWIWFlbH4b1n3vUET3P71QamHrF5itYyaqRU5pMZV3Cwwr81X8q7h5hCzm3Ip78RpMzfjEqTG0RcM2TLGccUrlkWVBLh4hn0NFpUIkzVFugFA5ooJffo-0AdJnO3mGWEOnXNVFWftJbK8cKnTns0DISrYFOyH_PlOe_YHCxgIXIT-dW8G-nbqoUjn5SBqunr36rcpaYCIwe2va4W_AcLCT43xiEAezRER_U9AuIqi_22KMd6SuHTyljhmWFPvPk8-gpjthLWXhcE7LPO5dV41hnZHnbI4n_9eI1nSVm7q9XdSvX1sWKV1GCwn8oj017AnxVvl9bScultko_0dTC747UqJ6UTFakLuFcHFe-F5Tz7ItDWlBVPoXeC7gTpyuicFKLsdqGlW9F5X6kIwNrBRj9uRsS-QuzSER-fVkQCn4dUTcokttRH_0bYvyfr9oqiDXmywMgOp-L0sKayk8JOVynh2q0Tju9sdkvFr0PxoAjhofomfIC1SZ_JkOzwAT1TUW8dLjPHluMct34xW_-qna1AmkoxM4bZQLhllap96NTC-0IdtzeKDrTul8p7u3WXSJjjEMSijibTNMlnkB0AluT1_RNO94OnzuFv4YlcV24FPhJzchhbyKREkOb_wzgcnSbRwGHjIcfRgkX-IzoXHVBcMYFUrPmsXrnRcfad4XwjkUOgvivkURW2_EwnzgrLDh-IKek51_0FpT1MnFCSG0gQbVSs_iMVPr6UXNAw62LGbKVtl3ZMXyapEpcO8azNbn6Wvd550R704JXxYlU)).

@lavalamp @krousey @smarterclayton

Automatic merge from submit-queue

Fix srv record lookup

Took longer than expected for unrelated reasons. This is intentionally a pretty dumb fix.
Fixes https://github.com/kubernetes/kubernetes/issues/26116

Automatic merge from submit-queue

volume controller: add configurable integration test to stress the binder

The test tries to bind configured nr. of PVs to the same nr. of PVCs. '100' is used by default, which should take ~1-3 seconds (depends on log level). Periodic sync is needed in rare cases, which may add another 10 seconds. - cache from #25881 will help here and sync should not be needed at all.

The test is configurable and may be reused to measure binder performance. Set KUBE_INTEGRATION_PERSISTENTVOLUME_* env. variables as described in persistent_volume_test.go and run the tests:

```
# compile
$ cd test/integration
$ godep go test -tags 'integration no-docker' -c

# run the tests
$ KUBE_INTEGRATION_PERSISTENTVOLUME_SYNC_PERIOD=10s KUBE_INTEGRATION_PERSISTENTVOLUME_OBJECTS=1000 time ./integration.test -test.run TestPersistentVolumeMultiPVsPVCs -v 2
```

Log level '2' is useful to get timestamps of various events like 'TestPersistentVolumeMultiPVsPVCs: start' and 'TestPersistentVolumeMultiPVsPVCs: claims are bound'.

Automatic merge from submit-queue

volume controller: Convert PersistentVolumes from Kubernetes 1.2

In Kubernetes 1.2 we used template PersistentVolume for provisioning. When a claim for dynamic volume was detected, Kubernetes did:

- create template PV for the claim with dummy pointer to storage asset
- allocate storage asset such as AWS EBS
- fill real pointer to the created storage asset to the template PV

In refactored volume provisioner, Kubernetes allocates the storage asset first and then creates a Kubernetes PV instance already with the correct pointer to the storage asset.

To support seamles upgrade from 1.2 to 1.3 we need to remove these unprovisioned template PVs. The new controller does not use them, it will see PVC for dynamic provisioning and create real PV instead.

See https://github.com/pmorie/pv-haxxz/pull/3 for pseudocode.

Automatic merge from submit-queue

Fix GCE attacher/detacher to ignore return value of failed calls.

The plugin should ignore any return value if err is set. Found when writing unit tests in #26615 - my dummy `DiskIsAttached` returned `false, errors.New('fake error')` and the volume was **not** detached although the log message `"Error checking if PD (%q) is already attached to current node (%q). Will continue and try detach anyway."` suggested otherwise 

@saad-ali, PTAL
@kubernetes/sig-storage 

Automatic merge from submit-queue

federation: Updating federation-controller-manager to use secret to get federation-apiserver's kubeconfig

Fixing the credentials problem: https://github.com/kubernetes/kubernetes/issues/26762#issuecomment-223690990.

Admin will create a secret with the name "federation-apiserver-secret" in the k8s cluster hosting the federation control plane. This secret will contain the kubeconfig to access federation-apiserver.
federation-controller-manager will use this secret to contact the federation-apiserver.
This flow is same as the one used by all federation-controllers to contact k8s apiservers that are part of the federation.

cc @kubernetes/sig-cluster-federation @lavalamp @erictune @colhom 

Automatic merge from submit-queue

kubelet e2e: enforce that image prepulling must finish before the test

The image prepulling pod calls docker directly to pull images. If the pod
hasn't finished before running the resource usage tracking test, there'd be a
cpu spike in docker. We'd rather wait and fail if this is the case, before
running the test.

Automatic merge from submit-queue

Link kernel log files to have them be copied over as part of node e2e.

Fixes #25993

cc @pwittrock 

Automatic merge from submit-queue

AWS Route53 dnsprovider

Still needs unit tests, and some other cleanup.  Review not urgent, but feel free to make a first pass.
Only need to look at the last two commits.  The prior commits will go in as #26020. This will need to be rebased against #26020 once that merges.
 
It's a bare minimum implementation, only what's required for Ubernetes Federated Services (managing basic A and CNAME records).  More functionality (health checks, geolocation etc) can be fairly easily added as required.

It also requires github.com/aws/aws-sdk-go/service/route53 to be vendored into godeps, which I haven't managed to do successfully yet (Oh Godep!)

cc: @justinsb FYI

Automatic merge from submit-queue

Wait for all volumes/claims to get synced in unit test.

Controller.HasSynced() returns true when all initial claims/volumes were sent
to appropriate goroutines, not when the goroutine has actually processed them.

Fixes #26712

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Signed-off-by: Vishnu kannan <vishnuk@google.com>