Automatic merge from submit-queue (batch tested with PRs 40943, 40967)

Switch kubectl version and api-versions to create a discovery client …

…directly.

The clientset will throw an error for aggregated apiservers because the
clientset looks for specific versions of apis that are compiled into
the client.  These will be missing from aggregated apiservers.
The discoveryclient is fully dynamic and does not rely on compiled
in apiversions.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 40943, 40967)

switch admission serialized config to an apiserver type

Switches the kube admission config from componentconfig to the new `apiserver.k8s.io` group so that all API servers can use the shared configuration.  This switch external serialization, but it does not move the code that reads the config.  I'd like to do that as a follow-on.

@kubernetes/sig-api-machinery-misc @kubernetes/api-reviewers @smarterclayton 
@derekwaynecarr ptal
@sttts

Automatic merge from submit-queue (batch tested with PRs 39681, 39321, 41018, 40883)

kubeadm: app/util/template.go has unit tests

**What this PR does / why we need it**: There was a TODO tag about adding unit tests, but unit tests have been added and the coverage for that file is ~90%.  Tag should be removed. 

Adding unit tests is a WIP from #34136

**Special notes for your reviewer**: /cc @luxas @pires 

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 39681, 39321, 41018, 40883)

Revert "Avoid unnecessary copies in cacher"

Reverts kubernetes/kubernetes#40735

Ref #41013

[I don't think it's related, but just in case it is...]

Automatic merge from submit-queue (batch tested with PRs 39681, 39321, 41018, 40883)

remove never used var

Automatic merge from submit-queue (batch tested with PRs 39681, 39321, 41018, 40883)

Avoid closing over range variables

Similar to #31053.

@justinsb since you authored the original one, ptal

Automatic merge from submit-queue

Discovery typo fix.

Simple typo fix.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

add liggitt to apimachinery/apiserver packages

Add @liggitt to apimachinery and apiserver packages.

Automatic merge from submit-queue (batch tested with PRs 40978, 40994, 41008, 40622)

Refactored kubemark code into provider-specific and provider-independent parts [Part-2]

Applying part of the changes of PR https://github.com/kubernetes/kubernetes/pull/39033 (which refactored kubemark code completely). The changes included in this PR are:

- Added test/kubemark/skeleton/util.sh which defines a well-commented interface that any cloud-provider should implement to run kubemark.
  This includes functions like creating the master machine instance along with its resources, remotely executing a given command on the master (like ssh), scp, deleting the master instance and its resources.
  All these functions have to be over-ridden by each cloud provider inside the file /test/kubemark/$CLOUD_PROVIDER/util.sh
- Implemented the above mentioned interface for gce in /test/kubemark/$CLOUD_PROVIDER/util.sh
- Made start- and stop- kubemark scripts (almost) provider independent by making them source the interface based on cloud provider.

@kubernetes/sig-scalability-misc @wojtek-t @gmarek

Automatic merge from submit-queue (batch tested with PRs 40978, 40994, 41008, 40622)

Fix failing density test in 100-node clusters

Automatic merge from submit-queue (batch tested with PRs 40978, 40994, 41008, 40622)

update kubeadm usage message

Automatic merge from submit-queue (batch tested with PRs 40978, 40994, 41008, 40622)

Add custom conversion funcs not directly into pkg/api.Scheme

Automatic merge from submit-queue

Kubeadm discovery remove error passing

**What this PR does / why we need it**: In the app/discovery there is some confusion about the passing of error values created in the discovery/token, discovery/https/ and discovery/file pkgs.  Since they always return `nil` , it was very confusing in discovery/flags.go why to propagate them up as if there was a chance for them to return a value other than `nil`.  This change makes it much more clear what is being passed.  

I noticed this as I was making a sweep through trying to add more unit tests and it was very confusing to read the code. 

**Which issue this PR fixes** : fixes #https://github.com/kubernetes/kubeadm/issues/141

**Special notes for your reviewer**: /cc @luxas @pires 

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue

kubeadm: Remove the kubernetes.io/cluster-service label from the Deployment templates

**What this PR does / why we need it**:

As discussed on Slack, these labels have no function when not using the addon-manager, so it's best to remove them to avoid confusion.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
@thockin @mikedanese @pires @MrHohn @bowei @dmmcquay @deads2k @philips

Automatic merge from submit-queue

Using API_HOST_IP to do apiserver health check.

In `hack/local-up-cluster.sh`, it's better to use `API_HOST_IP` to do apiserver health check.

Automatic merge from submit-queue

[Federation][kubefed] Add option to expose federation apiserver on nodeport service

**What this PR does / why we need it**:
This PR adds an option to kubefed to expose federation api server over nodeport. This can be useful to deploy federation in non-cloud environments. This PR is target to address #39271

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
[Federation] kubefed init learned a new flag, `--api-server-service-type`, that allows service type to be specified for the federation API server.
[Federation] kubefed init also learned a new flag, `--api-server-advertise-address`, that allows specifying advertise address for federation API server in case the service type is NodePort.
```

@kubernetes/sig-federation-misc @madhusudancs

Automatic merge from submit-queue

Add SIG to test owners

**What this PR does / why we need it**:
This PR adds a `sig` column to the test owners file generation script.

A problem experienced with the current owners file is that since members are auto-assigned there are times where tests are assigned to non-active users who don't follow up to notifications to fix flakes. By assigning a SIG to each test we can hold a group we know is active responsible for taking care of flakes it's less likely that flakes will fall through the cracks.

**Special notes for your reviewer**:
* A companion PR will go into *kubernetes/contrib* adding support for mungers parsing this new column.
   * Another PR in contrib will add labeling GitHub flake issues with the appropriate SIG
* Currently SIGs are not labeled, this will be added in another PR where SIG determinations can be discussed

@saad-ali @pwittrock

The clientset will throw an error for aggregated apiservers because the
clientset looks for specific versions of apis that are compiled into
the client.  These will be missing from aggregated apiservers.
The discoveryclient is fully dynamic and does not rely on compiled
in apiversions.

Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)

Extract util used by jsonmergepatch and SMPatch

followup  https://github.com/kubernetes/kubernetes/pull/40666#discussion_r99198931

Extract some util out of the `strategicMergePatch` to make `jsonMergePatch` doesn't depend on `strategicMergePatch`.

```release-note
None
```

cc: @liggitt

Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)

Rename experimental-cgroups-per-pod flag

**What this PR does / why we need it**:
1. Rename `experimental-cgroups-per-qos` to `cgroups-per-qos`
1. Update hack/local-up-cluster to match `CGROUP_DRIVER` with docker runtime if used.

**Special notes for your reviewer**:
We plan to roll this feature out in the upcoming release.  Previous node e2e runs were running with this feature on by default.  We will default this feature on for all e2es next week.

**Release note**:
```release-note
Rename --experiemental-cgroups-per-qos to --cgroups-per-qos
```

Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)

PV E2E: provide each spec with a fresh nfs host

**What this PR does / why we need it**:
PersistentVolume e2e currently reuses an NFS host pod created at the start of the suite and accessed by each test.  This is far less favorable than using a fresh volume per test.  Additionally, this guards against the volume host pod or it's kubelet being disrupted, which has led to flakes.

```release-note-none
```

Automatic merge from submit-queue (batch tested with PRs 40289, 40877, 40879, 39972, 40942)

Remove the temporary fix for pre-1.0 mirror pods

The fix was introduced to fix #15960 for pre-1.0 pods. It should be safe to remove
this fix now.

Automatic merge from submit-queue

fix --save-config in create subcommand

--save-config option doesn't work.
```
#kubectl create rolebinding a --role=b --user=c --save-config  
rolebinding "a" created
```
old output:
```
#kubectl get rolebinding a -o yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  creationTimestamp: 2017-01-23T10:09:48Z
  name: a
  namespace: default
  resourceVersion: "281"
  selfLink: /apis/rbac.authorization.k8s.io/v1beta1/namespaces/default/rolebindings/a
  uid: 12abccd6-e154-11e6-819c-525400f84377
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: b
subjects:
- apiVersion: rbac/v1alpha1
  kind: User
  name: c
```

expected output:
```
#kubectl get rolebinding a -o yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"kind":"RoleBinding","apiVersion":"rbac.authorization.k8s.io/v1beta1","metadata":{"name":"c","creationTimestamp":null},"subjects":[{"kind":"User","apiVersion":"rbac/v1alpha1","name":"c"}],"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"Role","name":"b"}}
  creationTimestamp: 2017-01-23T10:09:48Z
  name: a
  namespace: default
  resourceVersion: "281"
  selfLink: /apis/rbac.authorization.k8s.io/v1beta1/namespaces/default/rolebindings/a
  uid: 12abccd6-e154-11e6-819c-525400f84377
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: b
subjects:
- apiVersion: rbac/v1alpha1
  kind: User
  name: c
```

Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)

federation: Updating deletion helper to add both finalizers in a single update

Fixes https://github.com/kubernetes/kubernetes/issues/40837

cc @mwielgus @csbell

Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)

print apiserver log location on apiserver error

**What this PR does / why we need it**:
Improve user experience. Attempt to direct user to logs of failing component.

**Special notes for your reviewer**:
In addition to failure, point to logs so that a user can attempt to self remedy and have more information available to debug immediately. A user may not know that the failing component has logs.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)

allow the kubelet to request certificates

Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)

Add [Flaky] tag to persistent volumes tests

**What this PR does / why we need it**:
Persistent Volume tests continue to flake in CI.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 40906, 40924, 40938, 40902, 40911)

Check whether apiversions is empty

What this PR does / why we need it:
#39719 check whether apisversions get from /api is empty
Special notes for your reviewer:
@caesarxuchao

Automatic merge from submit-queue

Add an upgrade test for secrets.

**What this PR does / why we need it**: This PR adds an upgrade test for secrets. It creates a secret and makes sure that pods can consume it before an after an upgrade.

Automatic merge from submit-queue

Add Janet and Michail to test approver list

@kargakis @fejta

Automatic merge from submit-queue

CRI: Handle cri in-place upgrade

Fixes https://github.com/kubernetes/kubernetes/issues/40051.

## How does this PR restart/remove legacy containers/sandboxes?
With this PR, dockershim will convert and return legacy containers and infra containers as regular containers/sandboxes. Then we can rely on the SyncPod logic to stop the legacy containers/sandboxes, and the garbage collector to remove the legacy containers/sandboxes.

To forcibly trigger restart:
* For infra containers, we manually set `hostNetwork` to opposite value to trigger a restart (See [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/kuberuntime_manager.go#L389))
* For application containers, they will be restarted with the infra container.
## How does this PR avoid extra overhead when there is no legacy container/sandbox?
For the lack of some labels, listing legacy containers needs extra `docker ps`. We should not introduce constant performance regression for legacy container cleanup. So we added the `legacyCleanupFlag`:
* In `ListContainers` and `ListPodSandbox`, only do extra `ListLegacyContainers` and `ListLegacyPodSandbox` when `legacyCleanupFlag` is `NotDone`.
* When dockershim starts, it will check whether there are legacy containers/sandboxes.
  * If there are none, it will mark `legacyCleanupFlag` as `Done`.
  * If there are any, it will leave `legacyCleanupFlag` as `NotDone`, and start a goroutine periodically check whether legacy cleanup is done.
This makes sure that there is overhead only when there are legacy containers/sandboxes not cleaned up yet.

## Caveats
* In-place upgrade will cause kubelet to restart all running containers.
* RestartNever container will not be restarted.
* Garbage collector sometimes keep the legacy containers for a long time if there aren't too many containers on the node. In that case, dockershim will keep performing extra `docker ps` which introduces overhead.
  * Manually remove all legacy containers will fix this.
  * Should we garbage collect legacy containers/sandboxes in dockershim by ourselves? /cc @yujuhong 
* Host port will not be reclaimed for the lack of checkpoint for legacy sandboxes. https://github.com/kubernetes/kubernetes/pull/39903 /cc @freehan 

/cc @yujuhong @feiskyer @dchen1107 @kubernetes/sig-node-api-reviews 
**Release note**:

```release-note
We should mention the caveats of in-place upgrade in release note.
```

Automatic merge from submit-queue

Plumb subresource through subjectaccessreview

plumb all fields for subjectaccessreview into the resulting `authorizer.AttributesRecord`

```release-note
The SubjectAccessReview API passes subresource and resource name information to the authorizer to answer authorization queries.
```