Automatic merge from submit-queue

validate kube-apiserver options

**What this PR does / why we need it**:

Create Validate() or add more checks in existing Validate() for the following files:

* vendor/k8s.io/apiextensions-apiserver/pkg/cmd/server/start.go:80

* vendor/k8s.io/kube-aggregator/pkg/cmd/server/start.go:104

* vendor/k8s.io/sample-apiserver/pkg/cmd/server/start.go:82

* cmd/kube-apiserver/app/options/validation.go:49

**Which issue this PR fixes**: 

fixes #50301

**Special notes for your reviewer**:

This PR follows #50135 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Add a heap to client-go. Heap orders items with heap invariant ordering.

**What this PR does / why we need it**:
Heap is useful in implementing priority queues. Some components may need such ordering to process their highest priority objects first. Scheduler is going to be the first user of the heap. It will store pending pods ordered by their priority, so that the highest priority pods are popped first to be scheduled.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

ref/ #47604
ref/ #48646

@kubernetes/api-reviewers @kubernetes/sig-scheduling-pr-reviews @davidopp 
/assign @caesarxuchao

Automatic merge from submit-queue

Simplify a command for unmounting mounted directories

**What this PR does / why we need it**:
This PR simplifies a command for unmounting mounted directories. It replaces `cat | awk | grep` by a single `awk` expression.

**Special notes for your reviewer**:
Link about useless cat/grep usage: http://porkmail.org/era/unix/award.html#grep

**Credit**:
I saw this code fragment in the Julia Evans (@jvns) [write up](https://jvns.ca/blog/2017/07/30/a-couple-useful-ideas-from-google/).

**Release note**:
```release-note
NONE
```

CC @simo5

Automatic merge from submit-queue

Task 2: Added toleration to DaemonSet pods for node condition taints

**What this PR does / why we need it**:
If TaintByCondition was enabled, added toleration to DaemonSet pods for node condition taints.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #42001 

**Release note**:
```release-note
None
```

Automatic merge from submit-queue

remove the duplicate address of glusterfs

**What this PR does / why we need it**:
remove the duplicate endpoint address of glusterfs.
If there is no duplicate address of user config or the duplicate address is ok, we can remove the unused addr map.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```

Automatic merge from submit-queue

Typedef visitor to document parameters

adds a typedef to clarify the parameters of the visitor

updates the unit test to verify each namespace/name pair

Automatic merge from submit-queue

add fuzzer dir for each apigroup

Fixes https://github.com/kubernetes/kubernetes/issues/50081.

We should put fuzzer functions in each apigroups separately to make the code more modular instead of monolithic.

Automatic merge from submit-queue

code format in master_utils.go

**What this PR does / why we need it**:
code format

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A

**Release note**:
```release-note
None
```

Automatic merge from submit-queue

add some checks for fedration-apiserver options

**What this PR does / why we need it**:

I find there is a TODO, see https://github.com/kubernetes/kubernetes/blob/master/federation/cmd/federation-apiserver/app/options/validation.go#L30

This PR add some checks for fedration-apiserver options

@sttts 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Rewrite staging import verifier in Go
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

**What this PR does / why we need it**:

Maintaining Bash is nobody's idea of fun. The declarative config is infinitely easier to read, as well.

**Special notes for your reviewer**:

@sttts still trying to figure out the rules you have encoded in the shell script... very confusing. Please review the `kube-gen` specifically.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue

move logs to kubectl/util

Move `pkg/util/logs` to `pkg/kubectl/util/logs` per https://github.com/kubernetes/kubernetes/issues/48209#issuecomment-311730681
This will make kubeadm, kubefed, gke-certificates-controller and e2e have dependency on kubectl, which should be fine.

partially addresses: kubernetes/community#598

```release-note
NONE
```

/assign @apelisse @monopole

Automatic merge from submit-queue

Remove deprecated ESIPP beta annotations

**What this PR does / why we need it**:
Remove deprecated ESIPP beta annotations.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50187

**Special notes for your reviewer**:
/assign @MrHohn
/sig network

**Release note**:

```release-note
Beta annotations `service.beta.kubernetes.io/external-traffic` and `service.beta.kubernetes.io/healthcheck-nodeport` have been removed. Please use fields `service.spec.externalTrafficPolicy` and `service.spec.healthCheckNodePort` instead.
```

Automatic merge from submit-queue

GCE: filter IP by address when listing

Automatic merge from submit-queue

Added field CollisionCount to StatefulSetStatus

**What this PR does / why we need it**:
This PR added a new field `CollisionCount` into `StatefulSetStatus`, similarly in terms of both name and semantics to the existing `CollisionCount` field in `DaemonSetStatus`.  The field will be used for collision avoidance when the `StatefulSet` controller creates name for the newest ControllerRevision, which will be done in another PR.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: #49909.

**Special notes for your reviewer**:
A second PR will include logic that actually uses the field for collision avoidance.

**Release note**:
```release-note
Added field CollisionCount to StatefulSetStatus in both apps/v1beta1 and apps/v1beta2
```

Automatic merge from submit-queue

add newline for  unimplemented RequiredDuringSchedulingRequiredDuringExecution

**What this PR does / why we need it**:
Unimplemented `RequiredDuringSchedulingRequiredDuringExecution` should not be treated as  comments of `RequiredDuringSchedulingIgnoredDuringExecution`

* [pkg/api/types.go#L1937](https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go#L1937)

* [staging/src/k8s.io/api/core/v1/types.go#L2155](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/api/core/v1/types.go#L2155)

>  // NOT YET IMPLEMENTED. TODO: Uncomment field once it is implemented.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:fixes #50483 

**Special notes for your reviewer**:

* Add an empty line to separate them. 

/cc @aveshagarwal @bsalamat @gyliu513 @k82cn @timothysc

**Release note**:

```release-note
None
```

Automatic merge from submit-queue

FC volume plugin: Support WWID for volume identifier

**What this PR does / why we need it**:

This PR adds World Wide Identifier (WWID) parameter to FCVolumeSource as an unique volume identifier.

**Which issue this PR fixes**: fixes #48639 

**Special notes for your reviewer**:

/cc @rootfs @jsafrane @msau42 

**Release note**:

```
FC volume plugin: Support WWID for volume identifier
```

Automatic merge from submit-queue

Migrate to controller references helpers in meta/v1

**What this PR does / why we need it**:
This is a follow up for #48319 that migrates all method usages to new methods in meta/v1.

**Special notes for your reviewer**:
Looking at each commit individually might be easier.

**Release note**:
```release-note
NONE
```
/sig api-machinery
/kind cleanup

Automatic merge from submit-queue

Fix my incorrect username

My mistake - used goog username rather than github.

Again, this is for kubectl extraction, currently blocked by need for many approvers in, e.g. #48580 #48581 #47011, etc.

Automatic merge from submit-queue

kubeadm: enhanced TLS validation for token-based discovery in `kubeadm join`

**What this PR does / why we need it**:
This PR implements enhanced TLS validation for `kubeadm join` when using token-based TLS discovery. Without this enhancement, `kubeadm join` has some less-than-ideal security properties. Specifically, in the case where a bootstrap token is compromised, the attacker can impersonate the API server to newly bootstrapping clients ([more discussion in the design proposal](https://docs.google.com/document/d/1SP4P7LJWSA8vUXj27UvKdVEdhpo5Fp0QHNo4TXvLQbw/edit?ts=5971498a)).

The gist of this enhancement is to support public key pinning in the style of [RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4). When bootstrapping, `kubeadm` can now be configured with a whitelist of root CA public keys. It can then validate that the cluster it connects to is operated by the owner of one of those public keys.

These public key hashes are short enough that the entire `kubeadm join` command can still be copy-pasted relatively easily (not as easily as before, but ~160 characters). Using a public key hash rather than a hash over the entire certificate allows certificates to be reissued with updated expirations without invalidating existing key pins.

This change adds two new command line flags (and associated config parameters):
 - **`--discovery-token-ca-cert-hash sha256:<hash>`:**
   Validates that the cluster root CA has a public key fingerprint that matches one of the specified values. If this flag is not passed when token-based discovery is being used, a warning is printed. This warning will become an error in 1.9.
 - **`--discovery-token-unsafe-skip-ca-verification`:**
    Disables the warning message when no keys are pinned. In 1.9, this flag will be required _unless_ `--discovery-token-unsafe-skip-ca-verification` is used.

This is fully backwards compatible and client side (kubeadm) only. It will be a breaking change when the flag becomes required in v1.9.

This validation is done after and in addition to the existing bootstrap token signing/MAC mechanism.

#### Example from `kubeadm init`:
```
$ kubeadm init
[...]

You can now join any number of machines by running the following on each node
as root:

  kubeadm join --token a66ae0.1f8a5ed9a210e187 192.168.42.10:6443 --discovery-token-ca-cert-hash sha256:547c102383c0f26387b961b4e9b8f842dc07c074c8316f238dbcf5563fc3ac35
```

#### Example from `kubeadm join`:
```
$ kubeadm join --token a66ae0.1f8a5ed9a210e187 192.168.42.10:6443 --discovery-token-ca-cert-hash sha256:547c102383c0f26387b961b4e9b8f842dc07c074c8316f238dbcf5563fc3ac35
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "192.168.42.10:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.42.10:6443"
[discovery] Requesting info from "https://192.168.42.10:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.42.10:6443"
[discovery] Successfully established connection with API Server "192.168.42.10:6443"

Node join complete:
* Certificate signing request sent to master and response
  received.
* Kubelet informed of new secure connection details.

Run 'kubectl get nodes' on the master to see this machine join.
```

**Which issue this PR fixes**:
ref https://github.com/kubernetes/features/issues/130
fixes: https://github.com/kubernetes/kubeadm/issues/365

**Special notes for your reviewer**:
This was proposed and discussed briefly by SIG-cluster-lifecycle and SIG-auth. The design proposal is [in Google Docs](https://docs.google.com/document/d/1SP4P7LJWSA8vUXj27UvKdVEdhpo5Fp0QHNo4TXvLQbw/edit?ts=5971498a).

There is a documentation change needed to explain the security properties of `kubeadm join` with and without `--discovery-token-ca-cert-hash`. This page should be linked by to by the warning message when you don't pass either of the new flags (I have it pointing [here](https://kubernetes.io/docs/admin/kubeadm/#kubeadm-join) for now, which I think will be the right place). I will follow up with this documentation shortly.

**Release note**:
```release-note
kubeadm: added enhanced TLS validation for token-based discovery in `kubeadm join` using a new `--discovery-token-ca-cert-hash` flag.
```

/cc @luxas @jbeda @ericchiang

My mistake - used goog username rather than github.

kubeadm: generated deepcopy for `k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm` and `k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1alpha1`.

This change adds the `--discovery-token-ca-cert-hash` and `--discovery-token-unsafe-skip-ca-verification` flags for `kubeadm join` and corresponding fields on the kubeadm NodeConfiguration struct. These flags configure enhanced TLS validation for token-based discovery.

The enhanced TLS validation works by pinning the public key hashes of the cluster CA. This is done by connecting to the `cluster-info` endpoint initially using an unvalidated/unsafe TLS connection. After the cluster info has been loaded, parsed, and validated with the existing symmetric signature/MAC scheme, the root CA is validated against the pinned public key set. A second request is made using validated/safe TLS using the newly-known CA and the result is validated to make sure the same `cluster-info` was returned from both requests.

This validation prevents a class of attacks where a leaked bootstrap token (such as from a compromised worker node) allows an attacker to impersonate the API server.

This change also update `kubeadm init` to print the correct `--discovery-token-ca-cert-hash` flag in the example `kubeadm join` command it prints at the end of initialization.

This change adds a `k8s.io/kubernetes/cmd/kubeadm/app/util/pubkeypin` package which implements x509 public key pinning in the style of RFC7469. This is the public key hash format used by the new `kubeadm join --discovery-token-ca-cert-hash` flag.

Hashes are namespaced with a short type, with "sha256" being the only currently-supported format. Type "sha256" is a hex-encoded SHA-256 hash over the Subject Public Key Info (SPKI) object in DER-encoded ASN.1.

Also move the function to gce_addresses.go so that metrics can be
recorded for the call.

Automatic merge from submit-queue

Add Cluster Autoscaler scalability test suite

This suite is intended for manually testing Cluster Autoscaler on large clusters. It isn't supposed to be run automatically (at least for now).

It can be run on Kubemark (with #50440) with the following setup:
- start Kubemark with NUM_NODES=1 (as we require there to be exactly 1 replica per hollow-node replication controller in this setup)
- set kubemark-master machine type manually to appropriate type for the Kubemark cluster size. Maximum Kubemark cluster size reached in test run is defined by maxNodes constant, so for maxNodes=1000, please upgrade to n1-standard-32. Adjust if modifying maxNodes.
- start Cluster Autoscaler pod in the external cluster using image built from version with Kubemark cloud provider (release pending)
- for grabbing metrics from ClusterAutoscaler (with #50382), add "--include-cluster-autoscaler=true" parameter in addition to regular flags for gathering components' metrics/resource usage during e2e tests

cc @bskiba

Automatic merge from submit-queue

Make ClusterID required for AWS.

**What this PR does / why we need it**:
Makes ClusterID required for AWS and provides a flag to run in un-tagged mode

fixes #48954 


**Release note**:
```release-note
A cluster using the AWS cloud provider will need to label existing nodes and resources with a ClusterID or the kube-controller-manager will not start.  To run without a ClusterID pass --allow-untagged-cloud=true to the kube-controller-manager on startup.
```

Automatic merge from submit-queue (batch tested with PRs 45186, 50440)

Add functionality needed by Cluster Autoscaler to Kubemark Provider.

Make adding nodes asynchronous. Add method for getting target
size of node group. Add method for getting node group for node.
Factor out some common code.

**Release note**:
```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 45186, 50440)

Retry fed-svc creation on diff NodePort during e2e tests

**What this PR does / why we need it**:
Currently in federated end2end tests, the creation of services are
done with a randomize NodePort selection take is causing e2e test
flakes if the creation of a federated service failed if the port is
not available.
Now the util.CreateService(...) function is retrying to create the
service on different nodePort in case of error. The method retry until
success or all possible NodePorts have been tested and also failed.

**Which issue this PR fixes** 
fixes #44018

This PR adds World Wide Identifier (WWID) parameter to
FCVolumeSource as an unique volume identifier.

fixes #48639

Automatic merge from submit-queue (batch tested with PRs 50306, 49624)

Add daemonset to all categories

**What this PR does / why we need it**:
We could get daemonset resource by running command `kubectl get all`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fix https://github.com/kubernetes/kubernetes/issues/49620

Automatic merge from submit-queue (batch tested with PRs 50306, 49624)

simplify logic around LB deletion for servicecontroller

**What this PR does / why we need it**: simplify logic around LB deletio

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**: discovered when aswering to #50299

**Release note**:

```release-note
None
```

Make adding nodes asynchronous. Add method for getting target
size of node group. Add method for getting node group for node.
Factor out some common code.