Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)

Clean up pre-ControllerRef compatibility logic

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43323 

**Special notes for your reviewer**:
No
**Release note**:

```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)

'workqueue' modify 'work queue'

Automatic merge from submit-queue (batch tested with PRs 44447, 44456, 43277, 41779, 43942)

variable 'controller' collides with imported package name

variable 'controller' collides with imported package name,
 'controller'  modify 'ctrl'

Automatic merge from submit-queue

Fixing the kubernetes-e2e charm README to use the containers namespace.

**What this PR does / why we need it**: The deploy command is not correct in the README.

**Which issue this PR fixes**: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/251

**Special notes for your reviewer**: This is a text based change to the kubernetes-e2e charm README

**Release note**:
```release-note
NONE
```

A user pointed out the instructions in the kubernetes-e2e README were incorrect. Fixing that.

Automatic merge from submit-queue

Harden requirements for lost logs in GCL e2e load tests

cc @crassirostris

Automatic merge from submit-queue

add validate to resource in can-i

```
#kubectl auth can-i get xxx
the server doesn't have a resource type "xxx"
```

Automatic merge from submit-queue

Support completion for --cluster and --user

**What this PR does / why we need it**:

kubectl is supported completion for `--cluster` and `--user`.

```
$ kubectl --user=<tab>
--user=minikube  --user=user01

$ kubectl --cluster=<tab>
--cluster=cluster01  --cluster=minikube
```

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Add completion support for --namespace and --cluster to kubectl
```

Automatic merge from submit-queue

Add a test scenario for default node slector

Automatic merge from submit-queue

Fix some links in CHANGELOG

Automatic merge from submit-queue

add kubelet tests to verify host clean up

**What this PR does / why we need it**:
Increasingly we're seeing more failures in persistent volume e2e tests where pv tests are run in parallel with disruptive tests. The quick solution is to tag the pv tests as Flaky. This pr addresses one cause of the flakiness and adds a disruptive kubelet test.
Once this pr is shown to not produce flakes the [Flaky] tag for the "HostCleanup" tests will be removed in a separate pr.
+ Adds volume tests to _kubelet.go_ motivated by issues [31272](https://github.com/kubernetes/kubernetes/issues/31272) and [37657](https://github.com/kubernetes/kubernetes/issues/37657)
+ Addresses  reverted pr [41178](https://github.com/kubernetes/kubernetes/pull/41178) and negates the need for pr [41229](https://github.com/kubernetes/kubernetes/pull/41229)

**Which issue this PR fixes** 
Adds regression tests to cover issues: #31272 and #37657

**Special notes for your reviewer**:
It's possible that one of the new tests, which relies on the existence of _/usr/sbin/rpc.nfsd_ in the nfs-server pod, will not work in the GCI container env. If this turns out to be true then I will add a `SkipIfProviderIs("gke")` to the `It` block.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue

Fix crash in kubectl run

I observed the problem in k8s 1.5.1, but the code is still there, so fixing it.
Here's the old backtrace - happened due to network disconnection during `kubectl run`:

```
ivan4th@i4mac:~$ kubectl run --rm -it alpine1 --image=alpine:3.5 --restart=Never /bin/sh
Waiting for pod default/alpine1 to be running, status is Pending, pod ready: false
If you don't see a command prompt, try pressing enter.
/ # panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xe8706]

goroutine 1 [running]:
panic(0x11e0d80, 0xc420016090)
        /usr/local/go/src/runtime/panic.go:500 +0x1a1
k8s.io/kubernetes/pkg/kubectl/cmd.waitForPod.func2(0xc400000008, 0x14783a0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/run.go:432 +0x126
k8s.io/kubernetes/pkg/util/interrupt.(*Handler).Run(0xc420379230, 0xc4206c11f0, 0x0, 0x0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/interrupt/interrupt.go:103 +0x109
k8s.io/kubernetes/pkg/kubectl/cmd.waitForPod(0x208c8c0, 0xc4204832c0, 0xc4204395d0, 0x7, 0xc4204395a0, 0x7, 0x1477e10, 0xc4206e6040, 0x0, 0x0, ...)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/run.go:434 +0x49e
k8s.io/kubernetes/pkg/kubectl/cmd.waitForPodTerminated(0x208c8c0, 0xc4204832c0, 0xc4204395d0, 0x7, 0xc4204395a0, 0x7, 0x1c48020, 0xc42002c010, 0xc420124300, 0x0, ...)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/run.go:458 +0xea
k8s.io/kubernetes/pkg/kubectl/cmd.Run(0x1c61040, 0xc4203106e0, 0x1c47fe0, 0xc42002c008, 0x1c48020, 0xc42002c010, 0x1c48020, 0xc42002c018, 0xc42047e6c0, 0xc42006d200, ...)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/run.go:304 +0xe37
k8s.io/kubernetes/pkg/kubectl/cmd.NewCmdRun.func1(0xc42047e6c0, 0xc42006d200, 0x2, 0x6)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/run.go:98 +0x144
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute(0xc42047e6c0, 0xc42006d140, 0x6, 0x6, 0xc42047e6c0, 0xc42006d140)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:603 +0x439
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4204dbb00, 0xc420438e50, 0x1, 0x1)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:689 +0x367
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4204dbb00, 0xc4203106e0, 0x1c47fe0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:648 +0x2b
k8s.io/kubernetes/cmd/kubectl/app.Run(0x0, 0x0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/app/kubectl.go:38 +0xd5
main.main()
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/kubectl.go:26 +0x22
```

Automatic merge from submit-queue

kubeadm: don't add proxy flag to apiservers below v1.7

Automatic merge from submit-queue

add deregistration for paths

Aggregation and TPRs require the ability to de-register endpoints.  This adds the capability and makes it threadsafe.

Automatic merge from submit-queue

kubectl/cmd: remove a bunch of unused parameters

Found with github.com/mvdan/unparam.

**Release note**: NONE

Automatic merge from submit-queue

Add default information to CronJobSpec

This is requested followup to https://github.com/kubernetes/kubernetes/pull/41890

@smarterclayton @janetkuo ptal

Automatic merge from submit-queue

[e2e] Bump up pod deletion time for source pod IP test

From #44225.

Source pod IP e2e test is pretty flaky lately, and most of the failures seem to be timeout waiting for pod "kube-proxy-mode-detector" to disappear. Didn't found any other thing suspicious.

This PR bumps pod deletion timeout to DefaultPodDeletionTimeout, which is 3 minutes. Hopefully it will mitigate the flakes.

/assign @freehan 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Use EventTypeWarning instead of EventTypeNormal when failure

Automatic merge from submit-queue

Do etcd semver validation using posix only

this is a follow-up to https://github.com/kubernetes/kubernetes/pull/44352, can't use sort -V because not everybody has that
```release-note
NONE
```

Automatic merge from submit-queue

upgrade wordpress to 4.7.3 latest

see https://github.com/kubernetes/kubernetes/pull/42029

/release-note-none

Automatic merge from submit-queue

Bugfix: cloud-controller-manager routeController.run() block in WaitForCacheSync

**What this PR does / why we need it**:
cloud-controller-manager routeController.run not run in goroutine will block to  wait nodeListerSynced because sharedInformers not started


**Special notes for your reviewer**:

cloud-controller-manager routeController.run  should run in goroutine like the same code in kube-controller-manager,or it will block to  wait nodeListerSynced,but sharedInformers not startd.

```go
//controller-manager.go:197
func StartControllers(s *options.CloudControllerManagerServer, kubeconfig *restclient.Config, rootClientBuilder, clientBuilder controller.ControllerClientBuilder, stop <-chan struct{}, recorder record.EventRecorder, cloud cloudprovider.Interface) error {
	// Function to build the kube client object
	client := func(serviceAccountName string) clientset.Interface {
		return rootClientBuilder.ClientOrDie(serviceAccountName)
	}
	versionedClient := client("shared-informers")
	sharedInformers := informers.NewSharedInformerFactory(versionedClient, resyncPeriod(s)())
ocateNodeCIDRs && s.ConfigureCloudRoutes {
		if routes, ok := cloud.Routes(); !ok {
			glog.Warning("configure-cloud-routes is set, but cloud provider does not support routes. Will not configure cloud provider routes.")
		} else {
			routeController := routecontroller.New(routes, client("route-controller"), sharedInformers.Core().V1().Nodes(), s.ClusterName, clusterCIDR)
                       //should run in goroutine
----->		routeController.Run(stop, s.RouteReconciliationPeriod.Duration)
			time.Sleep(wait.Jitter(s.ControllerStartInterval.Duration, ControllerStartJitter))
		}
	} else {
		glog.Infof("Will not configure cloud provider routes for allocate-node-cidrs: %v, configure-cloud-routes: %v.", s.AllocateNodeCIDRs, s.ConfigureCloudRoutes)
	}

---> 	sharedInformers.Start(stop)

	select {}
}



//routecontroller.go:77
func (rc *RouteController) Run(stopCh <-chan struct{}, syncPeriod time.Duration) {
	defer utilruntime.HandleCrash()

	glog.Info("Starting the route controller")
       //will block
--->   	if !cache.WaitForCacheSync(stopCh, rc.nodeListerSynced) {
		utilruntime.HandleError(fmt.Errorf("timed out waiting for caches to sync"))
		return
	}

	go wait.NonSlidingUntil(func() {
		if err := rc.reconcileNodeRoutes(); err != nil {
			glog.Errorf("Couldn't reconcile node routes: %v", err)
		}
	}, syncPeriod, wait.NeverStop)
}
```

Automatic merge from submit-queue

Reduce replication_controller log spam

Decrease verbosity and reword 'Observed updated replication controller
...' now that the issue it was added for has been fixed.

This was originally added to debug #31981, and it was fixed back in September 2016.

cc @gmarek @wojtek-t @kargakis @eparis @smarterclayton

Automatic merge from submit-queue

dnsprovider: Avoid panic if fields are nil

The aws-sdk has some helper functions which should generally be used
whenever dereferencing an AWS provided pointer, in case the pointer is
nil, which would otherwise be a panic.

Issue https://github.com/kubernetes/kops/issues/2347

```release-note
dnsprovider: avoid panic if route53 fields are nil
```

Automatic merge from submit-queue

Add check for maxUnavailable in DaemonSet e2e tests

cc @kargakis @janetkuo

Automatic merge from submit-queue

local up dns defaults/Privileged defaults so that [Conformance] sets mostly pass on local clusters.

Fixes #43651 So that only 4 tests fail out of the box.

Automatic merge from submit-queue

Adding load balancer src cidrs to GCE cloudprovider

**What this PR does / why we need it**:
As of January 31st, 2018, GCP will be sending health checks and l7 traffic from two CIDRs and legacy health checks from three CIDS. This PR moves them into the cloudprovider package and provides a flag for override.

Another PR will need to be address firewall rule creation for external L4 network loadbalancing #40778

**Which issue this PR fixes**
Step one of #40778
Step one of https://github.com/kubernetes/ingress/issues/197

**Release note**:
```release-note
Add flags to GCE cloud provider to override known L4/L7 proxy & health check source cidrs
```

Automatic merge from submit-queue

bump github.com/robfig/cron

Fixes #41655.

@janetkuo ptal

Automatic merge from submit-queue

Make get-kube.sh work properly the "ci/latest" pointer

**What this PR does / why we need it**: this is a (late) followup from #36419, fixing a bug discovered in https://github.com/kubernetes/kubernetes/pull/36419#issuecomment-265679578.

Basically, `get-kube-binaries.sh` looks at `$KUBERNETES_RELEASE_URL`, but we weren't properly overriding it in `get-kube.sh` when downloading binaries from the CI release bucket. With this change, we set the variable correctly, and everything works:

```console
$ KUBERNETES_RELEASE=ci/latest ~/code/kubernetes/src/k8s.io/kubernetes/cluster/get-kube.sh
Downloading kubernetes release v1.7.0-alpha.0.2068+3a3dc827
  from https://dl.k8s.io/ci/v1.7.0-alpha.0.2068+3a3dc827e45426/kubernetes.tar.gz
  to /tmp/foo/kubernetes.tar.gz
Is this ok? [Y]/n

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0   1004      0 --:--:-- --:--:-- --:--:--  1006
100 6023k  100 6023k    0     0  10.9M      0 --:--:-- --:--:-- --:--:-- 10.9M
Unpacking kubernetes release v1.7.0-alpha.0.2068+3a3dc827
Kubernetes release: v1.7.0-alpha.0.2068+3a3dc827
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://dl.k8s.io/ci/v1.7.0-alpha.0.2068+3a3dc827e45426
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://dl.k8s.io/ci/v1.7.0-alpha.0.2068+3a3dc827e45426
Is this ok? [Y]/n

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    991      0 --:--:-- --:--:-- --:--:--   987
100  348M  100  348M    0     0  39.1M      0  0:00:08  0:00:08 --:--:-- 34.2M
               
md5sum(kubernetes-server-linux-amd64.tar.gz)=e71c9b48f6551797a74de2b83b501c44
sha1sum(kubernetes-server-linux-amd64.tar.gz)=688dcf567b60e27e3d9bf97436154543432768cf
               
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0   1019      0 --:--:-- --:--:-- --:--:--  1025
100 29.0M  100 29.0M    0     0  32.2M      0 --:--:-- --:--:-- --:--:-- 95.4M
               
md5sum(kubernetes-client-linux-amd64.tar.gz)=8e6a90298411ae5a0e943b1c0e182b1d
sha1sum(kubernetes-client-linux-amd64.tar.gz)=187a2d2c1c6ae1ead32ec4c1fa51f695223edaae
               
Extracting /tmp/foo/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /tmp/foo/kubernetes/platforms/linux/amd64
Add '/tmp/foo/kubernetes/client/bin' to your PATH to use newly-installed binaries.
Creating a kubernetes on gce...
...

```
**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Add prometheus metrics for age of stats used for evictions.

Completes #42923 

This PR adds metrics for evictions, and records how stale data used for evictions is.

cc @vishh @derekwaynecarr @kubernetes/sig-node-pr-reviews

Automatic merge from submit-queue

Re-enabled HPA upgrade e2e test

Re-enabled HPA upgrade e2e test.

```release-note
NONE
```

Automatic merge from submit-queue

Make the dockershim root directory configurable

Make the dockershim root directory configurable so things like
integration tests (e.g. in OpenShift) can run as non-root.

cc @sttts @derekwaynecarr @yujuhong @Random-Liu @kubernetes/sig-node-pr-reviews @kubernetes/rh-cluster-infra