Automatic merge from submit-queue

Update "kubectl get all" to display resource type as part of name

fixes #23838
release-note-none

When running "kubectl get all", or printing any output with mixed resource kinds, an additional column is added to the output with each resource's kind:

`kubectl get all --all-namespaces`
```
NAMESPACE     NAME                         DESIRED          CURRENT       AGE
default       rc/docker-registry-1         1                1             23h
testproject   rc/node-1                    0                0             2d
NAMESPACE     NAME                         CLUSTER-IP       EXTERNAL-IP   PORT(S)                 AGE
default       svc/docker-registry          172.30.36.42     <none>        5000/TCP                23h
default       svc/kubernetes               172.30.0.1       <none>        443/TCP,53/UDP,53/TCP   7d
testproject   svc/ruby-ex                  172.30.187.128   <none>        8080/TCP                6d
NAMESPACE     NAME                         READY            STATUS        RESTARTS                AGE
default       po/docker-registry-1-cpf8o   1/1              Running       1                       23h
```

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

kubectl: don't display an empty list when trying to get a single resource that isn't found

Return immediately when attempting to get a singular resource that isn't found, so that we avoid
printing out a List if the output format is something like json or yaml.

Before:

```
$ kubectl get pod/foo -o yaml
apiVersion: v1
items: []
kind: List
metadata: {}
pods "foo" not found
```

After:

```
$ kubectl get pod/foo -o yaml
pods "foo" not found
```

Fixes #28243 

@kubernetes/kubectl @kubernetes/rh-ux @smarterclayton @liggitt @deads2k @metral 

Automatic merge from submit-queue

Graceful deletion bumps object's generation

Fix. #24935

cc @caesarxuchao 

Automatic merge from submit-queue

Add version and flag logging to kube-dns.

Helps with #28358

Stopgap until we have a general solution for #27744

Automatic merge from submit-queue

Print kube-dns pod logs on federation e2e test failure.

Automatic merge from submit-queue

Fix node confomance test

Fixes https://github.com/kubernetes/kubernetes/issues/28255, https://github.com/kubernetes/kubernetes/issues/28250, https://github.com/kubernetes/kubernetes/issues/28341.

The main reason of the flake is that in the failed test expects the `PodPhase` to keep `Pending`. It did `Eventually` check and `Consistently` check for 5 seconds. However, the default `PodPhase` is `Pending`, when the check passes, the `PodStatus` could still be in default state.

After that, the test expects the container status to be `Waiting`, which may not be the case, because the default `ContainerStatuses` is empty, and the pod could still be in the default state.

This PR changes the test to ensure `ContainerStatuses` first and then check the `PodPhase` after that.

Mark P1 because the test fails relatively frequently and does block some PRs.

@pwittrock 

/cc @liangchenye @ncdc 
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

E2e tests to check whether cluster autoscaling scale down works when one node pool is not autoscaled

E2e tests to check whether cluster autoscaling scale down works when one node pool is not autoscaled.

Automatic merge from submit-queue

Allow specifying secret data using strings

This PR allows specifying non-binary data values in `Secret` objects as `"stringData":{"key":"string value"}`, in addition to the existing base64 []byte serializations in the `data` field.

On write, the keys and values in the `stringData` field are merged to the `data` map, overwriting any values already present in the `data` map. The move is one-way, the `stringData` field is never output when reading from the API.

A Secret could be created like this:
```
{
  "kind":"Secret",
  "apiVersion":"v1",
  "metadata":{"name":"mysecret"},
  "data":{
    "image":"<base64-encoded-jpg>"
  },
  "stringData":{
    "username": "myuser",
    "password": "mypassword"
  }
}
```
and when read from the API would look like this:
```
{
  "kind":"Secret",
  "apiVersion":"v1",
  "metadata":{"name":"mysecret",...},
  "data":{
    "image":"<base64-encoded-jpg>"
    "username": "bXl1c2Vy",
    "password": "bXlwYXNzd29yZA=="
  }
}
```

Automatic merge from submit-queue

fixed the typo causing incorrect comparison with cVer and cVer; now checking against sVer

fixes #25252

Automatic merge from submit-queue

Kubelet: Add the docker semver back.

Fixes #28221.
This PR:
1) Add the semver back https://github.com/kubernetes/kubernetes/pull/20020
2) Remove the code in https://github.com/kubernetes/kubernetes/pull/20827, because docker 1.8 is not officially supported now, and we want to deprecate it. https://github.com/kubernetes/kubernetes/pull/27208
3) Add a test for docker version comparison.

XRef https://github.com/kubernetes/kubernetes/pull/28223

@yujuhong 
/cc @ingvagabund 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Install pip in kubekins test image and install AWS cli when needed

Follow-on to https://github.com/kubernetes/test-infra/pull/66. This fixes AWS hopefully.

I haven't pushed the image yet.

cc @fejta 

Automatic merge from submit-queue

fix return value

Automatic merge from submit-queue

invalid arg during applying SELinux label

Tested with Ubuntu 16.04 with SELINUX support. Actually, after `chcon `with long label got: `chcon: failed to set type security context component to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Invalid argument`

Automatic merge from submit-queue

Use slices of items to clean up after tests

Fixes #27582.

We used to maintain a pointer variable for each process to kill after the
tests finish. @lavalamp suggested using a slice instead, which is a much
cleaner solution. This implements @lavalamp's suggestion and also extends
the idea to tracking directories that need to be removed after the tests finish.

This also means that we should no longer check for nil `killCmd`s inside
`func (k *killCmd) Kill() error {...}` (see #27582 and #27589). If a nil
`killCmd` makes it in there, something is bad elsewhere and we want to see
the nil pointer exception immediately.

Mentioning @timstclair and @euank wrt the original issue/PR. 

Automatic merge from submit-queue

Should set default value for --service-node-port-range flag before verifying

For the flag `--service-node-port-range` of kube-apiserver, we know that it defaults to `30000-32767` if not specified. But if we only pass the flag `--kubernetes-service-node-port` with a valid value between `30000-32767` when starting kube-apiserver, a fatal error will occurs as the last below. It means that service port range is not 30000-32767 but empty. The log is from code [DefaultAndValidateRunOptions-->ValidateRunOptions-->verifyServiceNodePort](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L580) where the flags are verified. 

After tracing the apiserver related code, we can find the call stack:
```
func main() {
	......
	s := options.NewAPIServer()
	......
	app.Run(s)
	......
}
```
In the `app.Run`, it calls [genericapiserver.DefaultAndValidateRunOptions(s.ServerRunOptions)](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L80). But the `--kubernetes-service-node-port` hasn't been defaulted before there, so it's empty. It's then defaulted in `app.Run`-->[master.New](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L276)-->[genericapiserver.New](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/master/master.go#L179)-->[setDefaults](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L338)-->[defaultServiceNodePortRange](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L281).

So, we have to set default value for `--kubernetes-service-node-port` in [NewServerRunOptions](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/options/server_run_options.go#L105), as is done for `--secure-port` and/or `--insecure-port`. The `NewServerRunOptions` will be called in options.[NewAPIServer](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/options/options.go#L50)().

Hope that I have described the issue clearly. Thanks!

```
root@vm:~# kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001 &
[2] 24629
root@vm:~# F0627 23:46:37.308726   24629 genericapiserver.go:580] Kubernetes service port range  doesn't contain 30001
goroutine 1 [running]:
k8s.io/kubernetes/vendor/github.com/golang/glog.stacks(0x44f2500, 0x0, 0x0, 0x0)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:766 +0xb8
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).output(0x44d2020, 0xc800000003, 0xc820238000, 0x438c73b, 0x13, 0x244, 0x0)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:717 +0x259
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).printf(0x44d2020, 0xc800000003, 0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:655 +0x1d4
k8s.io/kubernetes/vendor/github.com/golang/glog.Fatalf(0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:1145 +0x5d
k8s.io/kubernetes/pkg/genericapiserver.verifyServiceNodePort(0xc8202a8400)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:580 +0x1d5
k8s.io/kubernetes/pkg/genericapiserver.ValidateRunOptions(0xc8202a8400)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:605 +0x2f
k8s.io/kubernetes/pkg/genericapiserver.DefaultAndValidateRunOptions(0xc8202a8400)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:612 +0x4e
k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc820224fc0, 0x0, 0x0)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:80 +0x70
main.main()
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/apiserver.go:50 +0x121

[2]+  Exit 255                kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001
root@vm:~#

```

Automatic merge from submit-queue

Add printf formatting for rrset objects.

Without this you just get two pointers in the debug log.

Before:

```
I0627 21:48:44.136615       1 dns.go:215] Existing recordset {0xc820168830 0xc820691540} is not equal to needed recordset &{0xc820168848 0xc820686040}, removing existing and adding needed.
```

After:

```
I0627 22:26:46.221856       1 dns.go:215] Existing recordset                <(clouddns) "federated-service.e2e-tests-service-cuza5.federation.svc.us-central1-c.us-central1.kube.5yetis.net." type=CNAME rrdatas=["federated-service.e2e-tests-service-cuza5.federation.svc.us-central1.kube.5yetis.net."] ttl=180>
I0627 22:26:46.221885       1 dns.go:216] ... not equal to needed recordset <(clouddns) "federated-service.e2e-tests-service-cuza5.federation.svc.us-central1-c.us-central1.kube.5yetis.net." type=CNAME rrdatas=["federated-service.e2e-tests-service-cuza5.federation.svc.us-central1.kube.5yetis.net."] ttl=180>
I0627 22:26:46.221919       1 dns.go:217] ... removing existing and adding needed.
```

Automatic merge from submit-queue

Parallel waiting for pods in e2e

Fixes #27120 

Automatic merge from submit-queue

Describe container volume mounts

This patch will list the volume mounts defined for the container in `kubectl describe`:

```console
[root@localhost origin]# kubectl describe pods/ruby-ex-3-ehchv
Name:			ruby-ex-3-ehchv
Namespace:		test
Security Policy:	restricted
Node:			localhost/10.0.2.15
Start Time:		Mon, 27 Jun 2016 11:15:19 +0000
Labels:			app=ruby-ex
			deployment=ruby-ex-3
			deploymentconfig=ruby-ex
Status:			Running
IP:			172.17.0.3
Controllers:		ReplicationController/ruby-ex-3
Containers:
  ruby-ex:
    Container ID:	docker://75869f5dd5da39025ebfcf4cb970004d9cc99ee2f95524732f9102254b289b1e
    Image:		172.30.159.185:5000/test/ruby-ex@sha256:0faa1fcca255a269378a4db51b8eef2ae67312de8b21b033c46fe0ff414efaea
    Image ID:		docker://052fe9b4b929cc883fcbbf1e223373d2fbe8bdf407170210ee29a94c33b40cd3
    Port:		8080/TCP
    QoS Tier:
      cpu:		BestEffort
      memory:		BestEffort
    State:		Running
      Started:		Mon, 27 Jun 2016 11:15:20 +0000
    Ready:		True
    Restart Count:	0
    Volume Mounts:
      volume-eckcb:		/test
      default-token-uffs0:	/var/run/secrets/kubernetes.io/serviceaccount
    Environment Variables:	<none>
```

Without this you just get two pointers in the debug log.

Before:

I0627 21:48:44.136615       1 dns.go:215] Existing recordset {0xc820168830 0xc820691540} is not equal to needed recordset &{0xc820168848 0xc820686040}, removing existing and adding needed.

After:

I0627 22:26:46.221856       1 dns.go:215] Existing recordset                <(clouddns) "federated-service.e2e-tests-service-cuza5.federation.svc.us-central1-c.us-central1.kube.5yetis.net." type=CNAME rrdatas=["federated-service.e2e-tests-service-cuza5.federation.svc.us-central1.kube.5yetis.net."] ttl=180>
I0627 22:26:46.221885       1 dns.go:216] ... not equal to needed recordset <(clouddns) "federated-service.e2e-tests-service-cuza5.federation.svc.us-central1-c.us-central1.kube.5yetis.net." type=CNAME rrdatas=["federated-service.e2e-tests-service-cuza5.federation.svc.us-central1.kube.5yetis.net."] ttl=180>
I0627 22:26:46.221919       1 dns.go:217] ... removing existing and adding needed.

Automatic merge from submit-queue

Implement custom help command for kubectl

```release-note
* kubectl help now provides "Did you mean this?" suggestions for typo/invalid command names.
```

Custom implementation of help command allows to print `Did you mean this?` with
suggestions, which is missed in embed help command from github.com/spf13/cobra

Also, it can be extended with different search features. At this patch, help
command searches query in short descriptions of commands in case of mismatch
with commands names.

fixes #25234

Automatic merge from submit-queue

Federated Services e2e: Simplify logic and logging around verificatio…

Simplify logic and logging around verification of underlying services.

Fixes #28269.

Without this PR, service verification in 4 of our e2e tests sometimes fails.

Automatic merge from submit-queue

switch back to promiscuous-bridge mode

fix: #27498

Own implemenation of help command allows to print `Did you mean this?` with
suggestions, which is missed in embed help command from github.com/spf13/cobra

Also, it can be extended with different search features. At this patch, help
command search query in short descriptions of commands in case of mismatch
with commands names.

fixes #25234

Federated Services e2e: Simplify logic and logging around verification of services in underlying clusters.  Fixes #28269

Automatic merge from submit-queue

oidc auth plugin: don't hard fail if provider is unavailable

When using OpenID Connect authentication, don't cause the API
server to fail if the provider is unavailable. This allows
installations to run OpenID Connect providers after starting the
API server, a common case when the provider is running on the
cluster itself.

Errors are now deferred to the authenticate method.

cc @sym3tri @erictune @aaronlevy @kubernetes/sig-auth

Automatic merge from submit-queue

[kubelet] Allow opting out of automatic cloud provider detection in kubelet. By default kubelet will auto-detect cloud providers

fixes #28231

Make node e2e exit nonzero on test failures

Automatic merge from submit-queue

Fix problems with container restarts and flocker volumes

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

The issue is described in #22436. A pod with a flocker volume attached can't be restarted. A complete new pod has to be created, then everything is working fine.

The problem is around the `isReady` check. This ready state is not getting cleaned up for restarting containers. IMHO we don't need this ready state stored at all. Maybe it was meant to not call the flocker API during pod restarts, but I would rather have it removed... (happy about some hints why we need this lines)

I tested it in a AWS/flocker cluster: starting/restarting/rescheduling on another node (in the same zone) works perfectly fine.

Automatic merge from submit-queue

Bump skydns godeps to latest

Update Godeps for github.com/skynetservices/skydns and miekg/dns.

Bump kubedns version to 1.6 with latest skynetservices/skydns code
    
Built kube-dns for all architectures and pushed containers to gcr.io.