Automatic merge from submit-queue (batch tested with PRs 47416, 47408, 49697, 49860, 50162)

kubeadm: Replace *clientset.Clientset with clientset.Interface

**What this PR does / why we need it**:

Needed for https://github.com/kubernetes/kubernetes/pull/48899
We should always use `clientset.Interface` instead of `*clientset.Clientset`, for better testability and all the other benefits of using an interface.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
Should be straightforward to merge

**Release note**:

```release-note
NONE
```
@timothysc @dmmcquay @pipejakob

Automatic merge from submit-queue (batch tested with PRs 47416, 47408, 49697, 49860, 50162)

fix error message for cronjob

**What this PR does / why we need it**:
fix error message for cronjob
expected an error would be better than unexpected lack of error.
ref: https://github.com/kubernetes/kubernetes/blob/master/cmd/genutils/genutils_test.go#L33

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 47416, 47408, 49697, 49860, 50162)

add possibility to use multiple floatingip pools in openstack loadbalancer

**What this PR does / why we need it**: Currently only one floating pool is supported in kubernetes openstack cloud provider. It is quite big issue for us, because we want run only single kubernetes cluster, but we want that external and internal services can be used. It means that we need possibility to create services with internal and external pools.

**Which issue this PR fixes**: fixes #49147

**Special notes for your reviewer**: service labels is not maybe correct place to define this floatingpool id. However, I did not find any better place easily. I do not want start modifying service api structure.

**Release note**:
```release-note
Add possibility to use multiple floatingip pools in openstack loadbalancer
```

Example how it works:
```
cat /etc/kubernetes/cloud-config
[Global]
auth-url=https://xxxx
username=xxxx
password=xxxx
region=yyy
tenant-id=b23efb65b1d44b5abd561511f40c565d
domain-name=foobar

[LoadBalancer]
lb-version=v2
subnet-id=aed26269-cd01-4d4e-b0d8-9ec726c4c2ba
lb-method=ROUND_ROBIN
floating-network-id=56e523e7-76cb-477f-80e4-2dc8cf32e3b4
create-monitor=yes
monitor-delay=10s
monitor-timeout=2000s
monitor-max-retries=3
```
```
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: web
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: web-ext
  name: web-ext
  namespace: default
spec:
  selector:
    run: web
  ports:
  - port: 80
    name: https
    protocol: TCP
    targetPort: 80
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: web-int
    floatingPool: a2a84887-4915-42bf-aaff-2b76688a4ec7
  name: web-int
  namespace: default
spec:
  selector:
    run: web
  ports:
  - port: 80
    name: https
    protocol: TCP
    targetPort: 80
  type: LoadBalancer
```
```
% kubectl create -f example.yaml
deployment "nginx-deployment" created
service "web-ext" created
service "web-int" created
% kubectl get svc -o wide
NAME         CLUSTER-IP       EXTERNAL-IP                   PORT(S)        AGE       SELECTOR
kubernetes   10.254.0.1       <none>                        443/TCP        2m        <none>
web-ext      10.254.23.153    192.168.1.57,193.xx.xxx.xxx   80:30151/TCP   52s       run=web
web-int      10.254.128.141   192.168.1.58,10.222.130.80    80:32431/TCP   52s       run=web
```

cc @anguslees @k8s-sig-openstack-feature-requests @dims

Automatic merge from submit-queue (batch tested with PRs 47416, 47408, 49697, 49860, 50162)

follow our go code style: error->err

Fixes https://github.com/kubernetes/kubernetes/issues/50189
```release-note
NONE
```

Automatic merge from submit-queue

simplify if and else for code
Signed-off-by: allencloud <allen.sun@daocloud.io>

**What this PR does / why we need it**:
This PR tries to simplify the code of if and else, and this could make code a little bit cleaner.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
NONE

**Special notes for your reviewer**:
NONE

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Timeout and Max-in-flight don't report non-resource URLs correctly.

Unify error reporting for 429 and 504 to be correct for timeout and max in flight and eviction. Add better messages to eviction (removing a todo). Return the correct body content for timeouts (reason and code should be correct).

This potentially increases cardinality of 429, but because non-api urls may be under the max-inflight budget we need to report them somewhere (if something breaks and starts fetching API versions endlessly).

```release-note
The 504 timeout error was returning a JSON error body that indicated it was a 500.  The body contents now correctly report a 500 error.
```

Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

Add some table printers

Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

fix swallowed error in kubectl rolling_updater

This fixes a swallowed error in kubectl. 

AddDeploymentKeyToReplicationController() is already tested, but there was an error that was not being exposed.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

add label examples for kubectl run

**What this PR does / why we need it**:

Add `--labels` examples for kubectl run

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
xref #49854

**Special notes for your reviewer**:
/cc @ahmetb 

**Release note**:

```release-note
add examples for kubectl run --labels
```

Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

Enable overriding fluentd resources in GCP

**What this PR does / why we need it**: This enables overriding fluentd resources in GCP, when there is a need for custom ones.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48487, 49009, 49862, 49843, 49700)

validate cadvisor rootpath

**What this PR does / why we need it**:

When working on issue #48452, I found [KubeletConfiguration.RootDirectory](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L525) was never been validated. The default value is set to ["/var/lib/kubelet"](https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/componentconfig/v1alpha1/defaults.go#L342).

If this directory does not exist in the file system, the [cadvisor.manager](https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/google/cadvisor/manager/manager.go#L679) will fail to gather the information for metrics.

> error trying to get filesystem Device for dir /var/lib/kubelet: err: stat failed on /var/lib/kubelet with error: no such file or directory

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

/cc @feiskyer @k82cn

**Release note**:

```release-note
validate cadvisor rootpath
```

Automatic merge from submit-queue

go-client: Use httpcache client for all requests, even though only openapi returns ETags for caching

**What this PR does / why we need it**: Use HTTP ETag for caching Swagger spec download

This also adds a new command-line flag "cachedir" to specify where the cache should keep its file. It defaults to `$HOME/.kube/http-cache`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: partly #38637

**Special notes for your reviewer**:
Because this adds a bunch of dependencies, and removes a couple of files, I do recommend reading each commit individually.

**Release note**:
```release-note
```

Automatic merge from submit-queue

Update build requirements

**What this PR does / why we need it**:
Update the build requirements for docker so that the three configurations are shown as sub-lists instead of a parallel list. 

Currently, the requirements are shown in this way.
1. xxx
2. xxx
3. xxx
4. xxx
5. xxx

But it should be shown as something like this:
1. xxx
  * xxx
  * xxx
  * xxx
2. xxx

The problem is because markdown doesn't support well for ordered sub-list under ordered list. So, change it to unordered sub-list can avoid this problem.

Automatic merge from submit-queue

Update OWNERS to correct members' handles

**What this PR does / why we need it**:
Fix some typos of members' handles as per https://github.com/kubernetes/kubernetes/issues/50048#issuecomment-319831957.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Associated with: #50048

**Special notes for your reviewer**:
/cc @madhusudancs @sebgoa @liggitt @saad-ali

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Add information about staging repos to README

Pretty much what it sounds like.

Add a new command-line cachedir flag to specify where to store the http
cache responses. This cache will only be used for OpenAPI Swagger spec
for now (as this is the only end-point that returns an ETag).

The cache will be removed and replaced with HTTP Etag caching instead.
This patch is simply removing the existing mechanism.

Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)

Task 0: Added node taints labels and feature flags

**What this PR does / why we need it**:
Added node taint const for node condition.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #42001

**Release note**:
```release-note
None
```

Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)

Move remaining cert helper functions from pkg/serviceaccount to client-go/util/cert

**What this PR does / why we need it**:
Unifies all remaining certificate helper functions from pkh/serviceaccount to client-go/util/cert. Private key functions were moved in #40907

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #40807

**Special notes for your reviewer**:

**Release note**:

Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)

Fail on swap enabled and deprecate experimental-fail-swap-on flag

**What this PR does / why we need it**:

    * Deprecate the old experimental-fail-swap-on
    * Add a new flag fail-swap-on and set it to true

    Before this change, we would not fail when swap is on. With this
    change we fail for everyone when swap is on, unless they explicitly
    set --fail-swap-on to false.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes #34726

**Special notes for your reviewer**:

**Release note**:

```release-note
Kubelet will by default fail with swap enabled from now on. The experimental flag "--experimental-fail-swap-on" has been deprecated, please set the new "--fail-swap-on" flag to false if you wish to run with /proc/swaps on.
```

Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)

Add basic install and mount flexvolumes e2e tests

fixes https://github.com/kubernetes/kubernetes/issues/47010

These two tests install a skeleton "dummy" flex driver, attachable and non-attachable respectively, then test that a pod can successfully use the flex driver. They are labeled disruptive because kubelet and controller-manager get restarted as part of the flex install. IMO it's important to keep this install procedure as part of the test to isolate any bugs with the startup plugin probe code.

There is a bit of an ugly dependency on cluster/gce/config-test.sh because --flex-volume-plugin-dir must be set to a dir that's readable from controller-manager container and writable by the flex e2e test. The default path is not writable on GCE masters with read-only root so I picked a location that looks okay.

In the "dummy" drivers I trick kubelet into thinking there is a mount point by doing "mount -t tmpfs none ${MNTPATH} >/dev/null 2>&1", hope that is okay.

I have only tested on GCE and theoretically they may work on AWS but I don't think there is a need to test on multiple cloudproviders.

-->
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 50119, 48366, 47181, 41611, 49547)

increase the GC e2e test timeout

Fix https://github.com/kubernetes/kubernetes/issues/50047. 

The root cause is #50046. See log analysis in #50047. For now, we just increase the timeout.

Automatic merge from submit-queue

kube-gen: cut off dependencies

Only gengo is left and apimachinery/pkg/openapi. The later will go away when #48861 merges.

This is the preparation that we can use kube-gen also from apimachinery without circular dependencies.

Automatic merge from submit-queue

Fix incorrect call to 'bind' in scheduler

I previously submitted https://github.com/kubernetes/kubernetes/pull/49661 -- I'm not sure if that PR is too big or what, but this is an attempt at a smaller PR that makes progress on the same issue and is easier to review.

**What this PR does / why we need it**:

In this refactor (https://github.com/kubernetes/kubernetes/commit/ecb962e6585#diff-67f2b61521299ca8d8687b0933bbfb19R223) the scheduler code was refactored into separate `bind` and `assume` functions. When that happened, `bind` was called with `pod` as an argument. The argument to `bind` should be the assumed pod, not the original pod. Evidence that `assumedPod` is the correct argument bind and not `pod`: https://github.com/kubernetes/kubernetes/blob/80f26fa8a89ef5863cb19c71a620bb389d025166/plugin/pkg/scheduler/scheduler.go#L229-L234. (and it says `assumed` in the function signature for `bind`, even though it's not called with the assumed pod as an argument).

This is an issue (and causes #49314, where pods that fail to bind to a node get stuck indefinitely) in the following scenario:

1. The pod fails to bind to the node
2. `bind` calls `ForgetPod` with the `pod` argument
3. since `ForgetPod` is expecting the assumed pod as an argument (because that's what's in the scheduler cache), it fails with an error like `scheduler cache ForgetPod failed: pod test-677550-rc-edit-namespace/nginx-jvn09 state was assumed on a different node`
4. The pod gets lost forever because of some incomplete error handling (which I haven't addressed here in the interest of making a simpler PR)

In this PR I've fixed the call to `bind` and modified the tests to make sure that `ForgetPod` gets called with the correct argument (the assumed pod) when binding fails.

**Which issue this PR fixes**: fixes #49314

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue

Fix etcd migration for HA clusters

@jpbetz 

Fix #43600

Automatic merge from submit-queue

federation-controller/clustercontroller: fix race when updating data

Fix #49958

Updates for ClusterController's maps were made without locking which can lead
to race conditions.

This change adds a RWMutex to protect the data. We lock and unlock them
whenever the data is accessed to make sure that we dont slow down too much

Automatic merge from submit-queue

compact rules which has the same ResourceName

**What this PR does / why we need it**:
make compact rule more elegant.
Before
```
# ./cluster/kubectl.sh describe clusterrole system:kube-scheduler
Name:		system:kube-scheduler
Labels:		kubernetes.io/bootstrapping=rbac-defaults
Annotations:	rbac.authorization.kubernetes.io/autoupdate=true
PolicyRule:
  Resources			Non-Resource URLs	Resource Names		Verbs
  ---------			-----------------	--------------		-----
  bindings			[]			[]			[create]
  endpoints			[]			[]			[create]
  endpoints			[]			[kube-scheduler]	[delete]
  endpoints			[]			[kube-scheduler]	[get]
  endpoints			[]			[kube-scheduler]	[patch]
  endpoints			[]			[kube-scheduler]	[update]
  events			[]			[]			[create patch update]
  nodes				[]			[]			[get list watch]
  persistentvolumeclaims	[]			[]			[get list watch]
  persistentvolumes		[]			[]			[get list watch]
  pods				[]			[]			[get list watch]
  pods/binding			[]			[]			[create]
  pods/status			[]			[]			[update]
  replicasets.extensions	[]			[]			[get list watch]
  replicationcontrollers	[]			[]			[get list watch]
  services			[]			[]			[get list watch]
  statefulsets.apps		[]			[]			[get list watch]
```
After
```
# ./cluster/kubectl.sh describe clusterrole system:kube-scheduler
Name:		system:kube-scheduler
Labels:		kubernetes.io/bootstrapping=rbac-defaults
Annotations:	rbac.authorization.kubernetes.io/autoupdate=true
PolicyRule:
  Resources			Non-Resource URLs	Resource Names		Verbs
  ---------			-----------------	--------------		-----
  bindings			[]			[]			[create]
  endpoints			[]			[]			[create]
  endpoints			[]			[kube-scheduler]	[delete get patch update]
  events			[]			[]			[create patch update]
  nodes				[]			[]			[get list watch]
  persistentvolumeclaims	[]			[]			[get list watch]
  persistentvolumes		[]			[]			[get list watch]
  pods				[]			[]			[get list watch]
  pods/binding			[]			[]			[create]
  pods/status			[]			[]			[update]
  replicasets.extensions	[]			[]			[get list watch]
  replicationcontrollers	[]			[]			[get list watch]
  services			[]			[]			[get list watch]
  statefulsets.apps		[]			[]			[get list watch]
```
**Which issue this PR fixes**:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

kubelet: remove code for handling old pod/containers paths

**What this PR does / why we need it**:
This PR removes the code for handling the paths that has been deprecated for a long time.

**Release note**:

```release-note
NONE
```

CC @simo5

Automatic merge from submit-queue (batch tested with PRs 46685, 49863, 50098, 50070, 50096)

GCE: Bump GLBC version to 0.9.6

Closes #50095

**Release note**:
```release-note
GCE: Bump GLBC version to 0.9.6
```

Automatic merge from submit-queue (batch tested with PRs 46685, 49863, 50098, 50070, 50096)

Remove 0,1,3 from rand.String to avoid 'bad words'

**What this PR does / why we need it**:

Remove 0,1,3 from the runes to prevent accidental bad words profanity

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #49543

**Special notes for your reviewer**:
/cc @mml  @jamesawebb1  @spiffxp 

**Release note**:

```release-note
Remove 0,1,3 from rand.String, to avoid 'bad words'
```

Automatic merge from submit-queue (batch tested with PRs 46685, 49863, 50098, 50070, 50096)

apiextensions: fix data race in storage

Fixes data race in CRD storage.

Copy to a new map because we cannot write to storageMap without a race as it is used without locking elsewhere.

**Release note**:

```release-note
NONE
```

/cc @sttts

Automatic merge from submit-queue (batch tested with PRs 46685, 49863, 50098, 50070, 50096)

Cover equivalence cache tests in scheduler core

**What this PR does / why we need it**:

Finish the last part of equivalence class scheduling #17390 : complete unit tests.

Also, removed `InvalidateCachedPredicateItemForPod` which is unused in latest design.

```
godep go test -v equivalence_cache_test.go  equivalence_cache.go  -cover

PASS
coverage: 92.3% of statements
ok  	command-line-arguments	0.071s
```

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

fix err message in namespace_policy

**What this PR does / why we need it**:

**Which issue this PR fixes**: fixes #49541

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```