Automatic merge from submit-queue

Stabilize controller unit tests.

Remove test "5-1", it's flaky as it depends on order of execution of goroutines. When the controller starts, existing claim is enqueued as "initial sync event" and a new volume is enqueued to separate goroutine. It is not deterministic which goroutine processes its events first and there is no way how to tell that the claim event was processed.

Also, force resync of the controllers after the test to make sure all events are processed.

Fixes unit test flakes.
@kubernetes/sig-storage 

Automatic merge from submit-queue

Switch DNS addons from skydns to kubedns

Change GCI and trusty cluster-helper scripts to use kubedns instead of skydns.

Automatic merge from submit-queue

kubelet e2e: set cpu/memory limits for docker 1.11

Docker 1.11 consumes more memory. Bump the limit to fix the tests. Also add
new limits for the 100-pod resource usage tracking test.

This fixes #26495

Automatic merge from submit-queue

Add Controller field to OwnerReference

cc @davidopp 

Automatic merge from submit-queue

Run l7 controller on master 

Fixes https://github.com/kubernetes/kubernetes/issues/23663, needs https://github.com/kubernetes/contrib/pull/680
@roberthbailey @kubernetes/goog-cluster 

Automatic merge from submit-queue

AWS: ELB proxy protocol support via annotation service.beta.kubernetes.io/aws-load-balancer-proxy-protocol

This is a ~~work in progress~~ branch that adds support for the Proxy Protocol with Elastic Load Balancers. The proxy protocol is documented here: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt. It allows us to pass the "real ip" address of a client to pods behind services.

As it stands now, we create an ELB policy on the load balancer that enables the proxy protocol. We then enumerate each node port assigned to the load balancer and add our newly created policy to it. The manual process is documented here: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html


Right now, I’m looking to get some feedback on the approach before I dive too much deeper in the code. More precisely, I have questions regarding the following:

1) Right now I just check that a certain annotation exists on the service regardless of what its value is. Assuming we’re going to enable this feature via an annotation, what is the expected experience? This decision likely depends on the answers to the next questions.

2) Right now the implementation enables the proxy protocol on every ELB backend. The actual ELB API expects you to add the policy for each configured backend. Do we want the ability to configure the proxy protocol on a per service port basis? For example, if a service exposes TCP 80 and 443, would we want the ability to only enable the proxy protocol on port 443? Does this overcomplicate the implementation? If we wanted to go this direction we could do something like ...

```
{
  "service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "tcp:80,tcp:443"
}
```

3) I avoided this because I was concerned with scope creep and our organization doesn’t need it, but could/should our implementation be adjusted to just handle ELB policies in general? I hadn’t used the ELB API until I started working on this branch so I don’t know how realistic this is. I also don't know how common this use case is as our organization has used our own load balancing setup prior to Kubernetes. This page has a couple of examples at the bottom: http://docs.aws.amazon.com/cli/latest/reference/elb/create-load-balancer-policy.html

cc @justinsb

<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24569)
<!-- Reviewable:end -->

Docker 1.11 consumes more memory. Bump the limit to fix the tests. Also add
new limits for the 100-pod resource usage tracking test.

Automatic merge from submit-queue

Add missing $ in util.sh

cc: @piosz @fgrzadkowski @jszczepkowski 

Automatic merge from submit-queue

kubectl run --restart=Never creates pods

Fixes #24533.

@bgrant0607 @janetkuo ptal
/fyi @thockin

```release-note
* kubectl run --restart=Never creates pods
```
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Fix apiservers crashes

Ref #26563

Unified skydns templates using a simple underscore based template and
added transform sed scripts to transform into salt and sed yaml
templates

Moved all content out of cluster/addons/dns into build/kube-dns and
saltbase/salt/kube-dns

Automatic merge from submit-queue

Fix some gce-only tests to run on gke as well

Enable "Services should work after restarting apiserver [Disruptive]" and DaemonRestart tests, except the 2 that require master ssh access.

Move restart/upgrade related test helpers into their own file in framework package.

Add ELB proxy protocol support via the annotation
"service.beta.kubernetes.io/aws-load-balancer-proxy-protocol". This
allows servers like Nginx and Haproxy to retrieve the real IP address of
a remote client.

Automatic merge from submit-queue

Add LabelSelector to PersistentVolumeClaimSpec

Implements #25413.

@kubernetes/sig-storage @bgrant0607 @thockin @jsafrane @eparis 

Automatic merge from submit-queue

rkt: Fix docker auth config save directory to avoid race.

Fixes #https://github.com/kubernetes/kubernetes/issues/26117

cc @euank @sjpotter 

Automatic merge from submit-queue

Enable cluster autoscaler scale down in e2e test

cc: @piosz @jszczepkowski @fgrzadkowski 

Automatic merge from submit-queue

Allow to enable scale down in kube-up for gce

cc: @piosz @fgrzadkowski @jszczepkowski 

Automatic merge from submit-queue

Removed metrics api group

```release-note
Removed metrics api group
```
The group is empty and unused. Kubelet Metrics API is defined in Kubelet code. Master Metrics API is defined in Heapster. Removing to avoid the confusion.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Remove test "5-1", it's flaky as it depends on order of execution of
goroutines. When the controller starts, existing claim is enqueued as
"initial sync event" and a new volume is enqueued to separate goroutine.
It is not deterministic which goroutine processes its events first and
there is no way how to tell that the claim event was processed.

Also, force resync of the controllers after the test to make sure all
events are processed.

Automatic merge from submit-queue

Fixed check in kubectl autoscale.

```release-note
Fixed check in kubectl autoscale: cpu consumption can be higher than 100%.
```

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Fixed check in kubectl autoscale: cpu consumption can be higher than 100%. Fixes #25815.

Automatic merge from submit-queue

Remove debug from GCI cluster autoscaler setup function

cc: @piosz @fgrzadkowski 

Automatic merge from submit-queue

AllocateOrOccupyCIDR returs quickly

Fix #26511.

AllocateOrOccupy is called before adding a Node to NCs cache. It turns out that sending an update to API server can take too long which will make NC think that given Node does not exist and all Pods from it will be removed.

The fix is to move the long part of the call (updating NodeSpec) to a separate go-routine. To prevent overloading the server and huge number of retries we limit the number of concurrent Update calls.

cc @zmerlynn @davidopp 

Automatic merge from submit-queue

daemonset handle DeletedFinalStateUnknown

During an e2e run in OpenShift we ran into the DS controller panic when handling `DeletedFinalStateUnknown`.  This PR checks for `DeletedFinalStateUnknown` and queues the embedded object if it is a `DaemonSet`.

@mikedanese - would you mind taking a look?
@deads2k  

```
panic: interface conversion: interface is cache.DeletedFinalStateUnknown, not *extensions.DaemonSet

goroutine 4369 [running]:
k8s.io/kubernetes/pkg/controller/daemon.func·005(0x2f8a0c0, 0xc20b559680)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/daemon/controller.go:160 +0x50
k8s.io/kubernetes/pkg/controller/framework.ResourceEventHandlerFuncs.OnDelete(0xc20a0ae090, 0xc20a0ae0a0, 0xc20a0ae0b0, 0x2f8a0c0, 0xc20b559680)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/framework/controller.go:178 +0x41
k8s.io/kubernetes/pkg/controller/framework.(*ResourceEventHandlerFuncs).OnDelete(0xc20b8ebf20, 0x2f8a0c0, 0xc20b559680)
	<autogenerated>:25 +0xb5
k8s.io/kubernetes/pkg/controller/framework.func·001(0x2f8a280, 0xc20b5522e0, 0x0, 0x0)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/framework/controller.go:248 +0x4be
k8s.io/kubernetes/pkg/controller/framework.(*Controller).processLoop(0xc20bb727e0)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/framework/controller.go:122 +0x6f
k8s.io/kubernetes/pkg/controller/framework.*Controller.(k8s.io/kubernetes/pkg/controller/framework.processLoop)·fm()
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/framework/controller.go:97 +0x27
k8s.io/kubernetes/pkg/util/wait.func·001()
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/util/wait/wait.go:66 +0x61
k8s.io/kubernetes/pkg/util/wait.JitterUntil(0xc209f8cfb8, 0x3b9aca00, 0x0, 0xc2080543c0)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/util/wait/wait.go:67 +0x8f
k8s.io/kubernetes/pkg/util/wait.Until(0xc209f8cfb8, 0x3b9aca00, 0xc2080543c0)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/util/wait/wait.go:47 +0x4a
k8s.io/kubernetes/pkg/controller/framework.(*Controller).Run(0xc20bb727e0, 0xc2080543c0)
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/framework/controller.go:97 +0x1fb
created by k8s.io/kubernetes/pkg/controller/daemon.(*DaemonSetsController).Run
	/data/src/github.com/openshift/origin/Godeps/_workspace/src/k8s.io/kubernetes/pkg/controller/daemon/controller.go:212 +0xae
```
https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_check/1002/artifact/origin/artifacts/test-cmd/logs/openshift.log

Automatic merge from submit-queue

Fix comment typos in #25972

@ixdy 

Automatic merge from submit-queue

Some fixes to tests to support large clusters