Automatic merge from submit-queue

Move shell completion generation into 'kubectl completion' command

Remove static shell completion scripts from the repo and add `completion` command to kubectl:

```bash
$ source <(kubectl completion bash)
```

or

```bash
$ source <(kubectl completion zsh)
```

This makes maintenance easier because no static scripts must be generated and committed anymore in the repo.

Moreover, kubectl is self-contained again for the user including the latest completion code. I am thinking about the use-case of updating kubectl via gcloud (or some package manager). The completion code is always in-sync, without the need to download a `contrib/completion/bash/kubectl` file from github.

Opinions are welcome /cc @eparis @nak3 

Fixes https://github.com/openshift/origin/issues/5290

<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/23801)
<!-- Reviewable:end -->

Automatic merge from submit-queue

Add BatchClient into clientset adaption

@soltysh FYI

Automatic merge from submit-queue

Kubelet: Cache image history to eliminate the performance regression

Fix https://github.com/kubernetes/kubernetes/issues/25057.

The image history operation takes almost 50% of cpu usage in kubelet performance test. We should cache image history instead of getting it from runtime everytime.

This PR cached image history in imageStatsProvider and added unit test.

@yujuhong @vishh 
/cc @kubernetes/sig-node 

Mark v1.3 because this is a relatively significant performance regression.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Add -addr option to hack/lib/etcd.h

Added -addr to etcd hack script so it listens on the same address specified in the `host` variable.
Signed-off-by: André Martins <aanm90@gmail.com>

Automatic merge from submit-queue

Node e2e - Fix issue where error in scp is not return.  Fixes #26435



[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

fix strategy patch diff list issue

fixes #25585 

@janetkuo @pwittrock ptal.

Automatic merge from submit-queue

kubectl: cast scale errors to actual errors when deleting

Fixes some of the deployment reaper timeouts in e2e

@kubernetes/deployment @soltysh 

Automatic merge from submit-queue

Use ubuntu-slim to reduce size of the iperf:e2e image

from
```
gcr.io/google_containers/iperf     e2e      8b3cc7064090        5 weeks ago     737.9 MB
```
to
```
gcr.io/google_containers/iperf     e2e      204325491636        33 seconds ago  61.09 MB
```

related to https://github.com/kubernetes/kubernetes/pull/25784#issuecomment-221706886
ping @bprashanth 

Automatic merge from submit-queue

Do not call NewFlannelServer() unless flannel overlay is enabled

Ref: #26093 

This makes so kubelet does not warn the user that iptables isn't in PATH, although the user didn't enable the flannel overlay.

@vishh @freehan @bprashanth 

Automatic merge from submit-queue

pkg/kubectl: preallocate slice

Automatic merge from submit-queue

Add assert.NotNil for test case

I hardcode the `DefaultInterfaceName` from `eth0` to `eth-k8sdefault` at release 1.2.0,  in order to test my CNI plugins. When running the test, it panics and prints wrongly formatted messages as below.

In the test case `TestBuildSummary`, `containerInfoV2ToNetworkStats` will return `nil` if `DefaultInterfaceName` is not `eth0`. So maybe we should add `assert.NotNil` to the test case.

```
ok      k8s.io/kubernetes/pkg/kubelet/server    0.591s
W0523 03:25:28.257074    2257 summary.go:311] Missing default interface "eth-k8sdefault" for s%!(EXTRA string=node:FooNode)
W0523 03:25:28.257322    2257 summary.go:311] Missing default interface "eth-k8sdefault" for s%!(EXTRA string=pod:test0_pod1)
W0523 03:25:28.257361    2257 summary.go:311] Missing default interface "eth-k8sdefault" for s%!(EXTRA string=pod:test0_pod0)
W0523 03:25:28.257419    2257 summary.go:311] Missing default interface "eth-k8sdefault" for s%!(EXTRA string=pod:test2_pod0)
--- FAIL: TestBuildSummary (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x471817]

goroutine 16 [running]:
testing.func·006()
        /usr/src/go/src/testing/testing.go:441 +0x181
k8s.io/kubernetes/pkg/kubelet/server/stats.checkNetworkStats(0xc20806d3b0, 0x140bbc0, 0x4, 0x0, 0x0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/stats/summary_test.go:296 +0xc07
k8s.io/kubernetes/pkg/kubelet/server/stats.TestBuildSummary(0xc20806d3b0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/stats/summary_test.go:124 +0x11d2
testing.tRunner(0xc20806d3b0, 0x1e43180)
        /usr/src/go/src/testing/testing.go:447 +0xbf
created by testing.RunTests
        /usr/src/go/src/testing/testing.go:555 +0xa8b
```

Automatic merge from submit-queue

Setting TLS1.2 minimum because TLS1.0 and TLS1.1 are vulnerable

TLS1.0 is known as vulnerable since it can be downgraded to SSL
https://blog.varonis.com/ssl-and-tls-1-0-no-longer-acceptable-for-pci-compliance/

TLS1.1 can be vulnerable if cipher RC4-SHA is used, and in Kubernetes it is, you can check it with
`
openssl s_client -cipher RC4-SHA -connect apiserver.k8s.example.com:443
`

https://www.globalsign.com/en/blog/poodle-vulnerability-expands-beyond-sslv3-to-tls/

Test suites like Qualys are reporting this Kubernetes issue as a level 3 vulnerability, they recommend to upgrade to TLS1.2 that is not affected, quoting Qualys:

`
RC4 should not be used where possible. One reason that RC4 was still being used was BEAST and Lucky13 attacks against CBC mode ciphers in
SSL and
TLS. However, TLSv 1.2 or later address these issues.
`

Automatic merge from submit-queue

scheduler: remove unused random generator

The way scheduler selecting host has been changed to round-robin.
Clean up leftover.

Automatic merge from submit-queue

Fix panic when the namespace flag is not present

We don't set the namespace in OpenShift, so we need to check if the namespace flag is present.

Automatic merge from submit-queue

Change base64 -d flag to --decode flag to make it BSD compatible

cc @roberthbailey 

Fixes #25998

Automatic merge from submit-queue

Convert the font (ubuntu) to paths in SVG

Closes #26103

Tim, PTAL

Automatic merge from submit-queue

pkg/runtime: preallocate slice in unstructured.go

Automatic merge from submit-queue

use MkTmpDir instead of ioutil.TempDir in testing

fixes #20243

Automatic merge from submit-queue

Support per-test-environment ginkgo flags for node e2e tests to facilitate skipping miss behaving tests in PR builder

We had an issue today where some node e2e tests were timing out in the pr builder.  We want to be able to skip tests in the pr builder and leave them running in the CI if this happens again.


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Added Test Cases for Pod

Test case modified for Pod

Automatic merge from submit-queue

Various kubenet fixes (panics and bugs and cidrs, oh my)

This PR fixes the following issues:

1. Corrects an inverse error-check that prevented `shaper.Reset` from ever being called with a correct ip address
2. Fix an issue where `parseCIDR` would fail after a kubelet restart due to an IP being stored instead of a CIDR being stored in the cache.
3. Fix an issue where kubenet could panic in TearDownPod if it was called before SetUpPod (e.g. after a kubelet restart).. because of bug number 1, this didn't happen except in rare situations (see 2 for why such a rare situation might happen)

This adds a test, but more would definitely be useful.
The commits are also granular enough I could split this up more if desired.

I'm also not super-familiar with this code, so review and feedback would be welcome.

Testing done:
```
$ cat examples/egress/egress.yml
 apiVersion: v1
kind: Pod
metadata:
  labels:
    name: egress
  name: egress-output
  annotations: {"kubernetes.io/ingress-bandwidth": "300k"}
spec:
  restartPolicy: Never
  containers:
    - name: egress
      image: busybox
      command: ["sh", "-c", "sleep 60"]
$ cat kubelet.log
...
Running: tc filter add dev cbr0 protocol ip parent 1:0 prio 1 u32 match ip dst 10.0.0.5/32 flowid 1:1
# setup
...
Running: tc filter del dev cbr0 parent 1:proto ip prio 1 handle 800::800 u32
# teardown
```

I also did various other bits of manual testing and logging to hunt down the panic and other issues, but don't have anything to paste for that 

cc @dcbw @kubernetes/sig-network 

Automatic merge from submit-queue

Fix quantity.CanonicalizeBytes docstring name

Automatic merge from submit-queue

Add a NodeCondition "NetworkUnavaiable" to prevent scheduling onto a node until the routes have been created 

This is new version of #26267 (based on top of that one).

The new workflow is:
- we have an "NetworkNotReady" condition
- Kubelet when it creates a node, it sets it to "true"
- RouteController will set it to "false" when the route is created
- Scheduler is scheduling only on nodes that doesn't have "NetworkNotReady ==true" condition

@gmarek @bgrant0607 @zmerlynn @cjcullen @derekwaynecarr @danwinship @dcbw @lavalamp @vishh 

Automatic merge from submit-queue

fixing docker file, and bumping cassandra version

- had issue with current v9 build.  `cassandra.yaml` was not editable by the cassandra user, chown and chmod is now correct
- rebuild jar against 3.5 cassandra version and bumped C* version

Automatic merge from submit-queue

rkt: Fix panic in setting ReadOnlyRootFS

What the title says. I wish this method were broken out in a reasonably unit testable way. fixing this panic is more important for the second though, testing will come in a later commit.

I observed the panic in a `./hack/local-up-cluster.sh` run with rkt as the container runtime.

This is also the panic that's failing our jenkins against master ([recent run](https://console.cloud.google.com/m/cloudstorage/b/rktnetes-jenkins/o/logs/kubernetes-e2e-gce/1946/artifacts/jenkins-e2e-minion-group-qjh3/kubelet.log for the log output of a recent run))

cc @tmrts @yifan-gu 

To avoid slice growth with append operations.

Automatic merge from submit-queue

Add garbage collector into kube-controller-manager

It's disabled by default.

Design doc is at https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/garbage-collection.md

Automatic merge from submit-queue

Add federation cluster status zone update

Hi Nikhil

Here's the cluster zone update stuff we discussed today.

What's remaining:

1. Rebase against #26308 (as soon as it merges) to expose the zones and region fields of cluster status.
2. Get hold of a cluster client via which to query nodes in the cluster. Currently we only have a DiscoveryClient, which doesn't know how to list nodes.

Any advice on the latter much appreciated.  Specifically how do I get a cluster client from a discovery client?

Q

Automatic merge from submit-queue

scheduler cache: print err if AssumePod fail

ref:
* https://github.com/kubernetes/kubernetes/issues/19681#issuecomment-222337964
* #26043

Print error to help debug flake.

Automatic merge from submit-queue

Sort revisions in rollout history as integers

Previously keys were sorted as strings, thus it was possible to see such order as 1, 10, 2, 3, 4, 5.

fixes: #25788 

Revert "Make Privileged pods node e2e use the framework"

Automatic merge from submit-queue

Make Privileged pods node e2e use the framework

Made the test more readable along the way with more logs. This should help us triage failures/flakes in the future.

#24577

Automatic merge from submit-queue

kubectl: move printObjectSpecificMessage in factory

@smarterclayton @kubernetes/kubectl PTAL

Automatic merge from submit-queue

Kubelet: Periodically reporting image pulling progress in log

Addresses https://github.com/kubernetes/kubernetes/issues/26075#issuecomment-221129896 and https://github.com/kubernetes/kubernetes/pull/26122#issuecomment-221128397.

This PR changes kube_docker_client to log pulling progress every *10* seconds. We can't print all progress messages into the log, because there are too many. So I make it report newest progress every 10 seconds to reduce log spam.
If the image pulling is too slow or stuck, we'll see image pulling progress unchanged or changed little overtime.

The following is the result if I set the reporting interval to *1* second.
```
I0524 00:53:26.189086  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "17b6a9e179d7: Pulling fs layer "
I0524 00:53:27.189082  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "202e40f8bb3a: Download complete "
I0524 00:53:28.189160  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Downloading [=>                                                 ] 1.474 MB/48.35 MB"
I0524 00:53:29.189156  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Downloading [====>                                              ] 3.931 MB/48.35 MB"
I0524 00:53:30.189089  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Downloading [=========>                                         ] 8.847 MB/48.35 MB"
I0524 00:53:31.189089  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Downloading [==================>                                ] 18.19 MB/48.35 MB"
I0524 00:53:32.189076  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Downloading [=======================================>           ] 38.34 MB/48.35 MB"
I0524 00:53:33.189106  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Extracting [=============>                                     ] 12.78 MB/48.35 MB"
I0524 00:53:34.189067  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Extracting [==================================>                ] 33.42 MB/48.35 MB"
I0524 00:53:35.189083  145099 kube_docker_client.go:252] Pulling image "ubuntu:latest": "487bffc61de6: Extracting [==================================================>] 48.35 MB/48.35 MB"
I0524 00:53:35.376667  145099 kube_docker_client.go:254] Finish pulling image "ubuntu:latest": "Status: Downloaded newer image for ubuntu:latest"
```

Ref image pulling related issue #19077.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

@yujuhong @dchen1107 
/cc @kubernetes/sig-node 

Automatic merge from submit-queue

rkt: Pass through podIP

This is needed for the /etc/hosts mount and the downward API to work.
Furthermore, this is required for the reported `PodStatus` to be
correct.

The `Status` bit mostly worked prior to #25062, and this restores that
functionality in addition to the new functionality.

In retrospect, the regression in status is large enough the prior PR should have included at least some of this; my bad for not realizing the full implications there.

#25902 is needed for downwards api stuff, but either merge order is fine as neither will break badly by itself.

cc @yifan-gu @dcbw