Automatic merge from submit-queue (batch tested with PRs 41919, 41149, 42350, 42351, 42285)

kubelet: enable qos-level memory limits

```release-note
Experimental support to reserve a pod's memory request from being utilized by pods in lower QoS tiers.
```

Enables the QoS-level memory cgroup limits described in https://github.com/kubernetes/community/pull/314

**Note: QoS level cgroups have to be enabled for any of this to take effect.**

Adds a new `--experimental-qos-reserved` flag that can be used to set the percentage of a resource to be reserved at the QoS level for pod resource requests.

For example, `--experimental-qos-reserved="memory=50%`, means that if a Guaranteed pod sets a memory request of 2Gi, the Burstable and BestEffort QoS memory cgroups will have their `memory.limit_in_bytes` set to `NodeAllocatable - (2Gi*50%)` to reserve 50% of the guaranteed pod's request from being used by the lower QoS tiers.

If a Burstable pod sets a request, its reserve will be deducted from the BestEffort memory limit.

The result is that:
- Guaranteed limit matches root cgroup at is not set by this code
- Burstable limit is `NodeAllocatable - Guaranteed reserve`
- BestEffort limit is `NodeAllocatable - Guaranteed reserve - Burstable reserve`

The only resource currently supported is `memory`; however, the code is generic enough that other resources can be added in the future.

@derekwaynecarr @vishh

Automatic merge from submit-queue (batch tested with PRs 41919, 41149, 42350, 42351, 42285)

Juju: Disable anonymous auth on kubelet

**What this PR does / why we need it**:

This disables anonymous authentication on kubelet when deployed via Juju.

I've also adjusted a few other TLS options for kubelet and kube-apiserver. The end result is that:
1. kube-apiserver can now authenticate with kubelet
2. kube-apiserver now verifies the integrity of kubelet

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:

https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/219

**Special notes for your reviewer**:

This is dependent on PR #41251, where the tactics changes are being merged in separately.

Some useful pages from the documentation:
* [apiserver -> kubelet](https://kubernetes.io/docs/admin/master-node-communication/#apiserver---kubelet)
* [Kubelet authentication/authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/)

**Release note**:

```release-note
Juju: Disable anonymous auth on kubelet
```

Automatic merge from submit-queue

Critial pod test uses allocatable instead of capacity

This solves #42239.

When this test was first introduced, pods could request up to the capacity of the node.
With the addition of allocatable introduced in #41234, this is no longer the case, and pods can only use up to allocatable.

This should be included in 1.6, as it is a bug related to a 1.6 feature.

cc @vish @yujuhong

Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)

Bump test timeouts to make secret tests work in large clusters

Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)

discovery restmapping should always prefer /v1

The core kube API, empty group, version==v1 should always be the most preferred group and resource from a rest mapper.  This special cases that.  All the others should be based on discovery order as we previously agreed.

@kubernetes/sig-cli-pr-reviews @kubernetes/sig-api-machinery-pr-reviews 
@enj

Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)

remove support for debian masters in GCE

Asked about this on the mailing list and no one objects.

@zmerlynn @roberthbailey 

```release-note
Remove support for debian masters in GCE kube-up.
```

Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)

Server timeout returns an incorrect error

Not a valid Status object in JSON

Part of #42163

Automatic merge from submit-queue (batch tested with PRs 41306, 42187, 41666, 42275, 42266)

Implement bulk polling of volumes

This implements Bulk volume polling using ideas presented by
justin in https://github.com/kubernetes/kubernetes/pull/39564

But it changes the implementation to use an interface
and doesn't affect other implementations.

cc @justinsb

Automatic merge from submit-queue (batch tested with PRs 42365, 42429, 41770, 42018, 35055)

Make Downward API test table-driven

Automatic merge from submit-queue (batch tested with PRs 42365, 42429, 41770, 42018, 35055)

kubeadm: Add --cert-dir, --cert-altnames instead of --api-external-dns-names

**What this PR does / why we need it**:

 - For the beta kubeadm init UX, we need this change
 - Also adds the `kubeadm phase certs selfsign` command that makes the phase invokable independently

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

This PR depends on https://github.com/kubernetes/kubernetes/pull/41897

**Release note**:

```release-note
```
@dmmcquay @pires @jbeda @errordeveloper @mikedanese @deads2k @liggitt

Automatic merge from submit-queue (batch tested with PRs 42365, 42429, 41770, 42018, 35055)

Updated scheduler name for multi-scheduler.

fixes #41859

Automatic merge from submit-queue (batch tested with PRs 42365, 42429, 41770, 42018, 35055)

controller: statefulsets respect observed generation

StatefulSets do not update ObservedGeneration even though the API field is in place. This means that clients can never be sure whether the StatefulSet controller has observed the latest spec of a StatefulSet.

@kubernetes/sig-apps-bugs

Automatic merge from submit-queue (batch tested with PRs 42365, 42429, 41770, 42018, 35055)

Remove kube-dns ConfigMap modification code from federation-{up,down}.sh scripts

PR #39338 has merged now. This shouldn't be necessary. This unbreaks federation tests.

```release-note
NONE
```

Handle potential conflicts on service endpoint test deletion

Cleanup should ignore conflicts and notfound errors

e2e/framework: Deflake cleanup of RCs in service_util.

The previous Get/Update pattern with no retry on resource version mismatch
would flake with the following error:

"the object has been modified; please apply your changes to the latest
version and try again"

Automatic merge from submit-queue

Remove defaults from string flags

- The default is printed automatically
 - The string text did not match the actual default

**What this PR does / why we need it**:
Adjust the documentation for flags on `client-gen`.

**Special notes for your reviewer**:
Doc change. String text only.

**Release note**:
```release-note
NONE
```

Before:
```
client-gen  --help
Usage of ./client-gen:
      --build-tag string                       A Go build tag to use to identify files generated by this command. Should be unique. (default "ignore_autogenerated")
      --clientset-api-path string              the value of default API path.
  -n, --clientset-name string                  the name of the generated clientset package. (default "internalclientset")
      --clientset-only                         when set, client-gen only generates the clientset shell, without generating the individual typed clients
      --clientset-path string                  the generated clientset will be output to <clientset-path>/<clientset-name>. Default to "k8s.io/kubernetes/pkg/client/clientset_generated/" (default "k8s.io/kubernetes/pkg/client/clientset_generated/")
      --fake-clientset                         when set, client-gen will generate the fake clientset that can be used in tests (default true)
  -h, --go-header-file string                  File containing boilerplate header text. The string YEAR will be replaced with the current 4-digit year. (default "/Users/mhb/go/src/k8s.io/gengo/boilerplate/boilerplate.go.txt")
      --included-types-overrides stringSlice   list of group/version/type for which client should be generated. By default, client is generated for all types which have genclient=true in types.go. This overrides that. For each groupVersion in this list, only the types mentioned here will be included. The default check of genclient=true will be used for other group versions.
      --input stringSlice                      group/versions that client-gen will generate clients for. At most one version per group is allowed. Specified in the format "group1/version1,group2/version2...". Default to "api/,extensions/,autoscaling/,batch/,rbac/" (default [api/,authentication/,authorization/,autoscaling/,batch/,certificates/,extensions/,rbac/,storage/,apps/,policy/])
      --input-base string                      base path to look for the api group. Default to "k8s.io/kubernetes/pkg/apis" (default "k8s.io/kubernetes/pkg/apis")
  -i, --input-dirs stringSlice                 Comma-separated list of import paths to get input types from.
  -o, --output-base string                     Output base; defaults to $GOPATH/src/ or ./ if $GOPATH is not set. (default "/Users/mhb/go/src")
  -O, --output-file-base string                Base name (without .go suffix) for output files.
  -p, --output-package string                  Base package path.
  -t, --test                                   set this flag to generate the client code for the testdata
      --verify-only                            If true, only verify existing output, do not write anything.
```
After:
```
client-gen  --help
Usage of ./client-gen:
      --build-tag string                       A Go build tag to use to identify files generated by this command. Should be unique. (default "ignore_autogenerated")
      --clientset-api-path string              the value of default API path.
  -n, --clientset-name string                  the name of the generated clientset package. (default "internalclientset")
      --clientset-only                         when set, client-gen only generates the clientset shell, without generating the individual typed clients
      --clientset-path string                  the generated clientset will be output to <clientset-path>/<clientset-name>. (default "k8s.io/kubernetes/pkg/client/clientset_generated/")
      --fake-clientset                         when set, client-gen will generate the fake clientset that can be used in tests (default true)
  -h, --go-header-file string                  File containing boilerplate header text. The string YEAR will be replaced with the current 4-digit year. (default "/Users/mhb/go/src/k8s.io/gengo/boilerplate/boilerplate.go.txt")
      --included-types-overrides stringSlice   list of group/version/type for which client should be generated. By default, client is generated for all types which have genclient=true in types.go. This overrides that. For each groupVersion in this list, only the types mentioned here will be included. The default check of genclient=true will be used for other group versions.
      --input stringSlice                      group/versions that client-gen will generate clients for. At most one version per group is allowed. Specified in the format "group1/version1,group2/version2...". (default [api/,authentication/,authorization/,autoscaling/,batch/,certificates/,extensions/,rbac/,storage/,apps/,policy/])
      --input-base string                      base path to look for the api group. (default "k8s.io/kubernetes/pkg/apis")
  -i, --input-dirs stringSlice                 Comma-separated list of import paths to get input types from.
  -o, --output-base string                     Output base; defaults to $GOPATH/src/ or ./ if $GOPATH is not set. (default "/Users/mhb/go/src")
  -O, --output-file-base string                Base name (without .go suffix) for output files.
  -p, --output-package string                  Base package path.
  -t, --test                                   set this flag to generate the client code for the testdata
      --verify-only                            If true, only verify existing output, do not write anything.
```

This implements Bulk volume polling using ideas presented by
justin in https://github.com/kubernetes/kubernetes/pull/39564

But it changes the implementation to use an interface
and doesn't affect other implementations.

Automatic merge from submit-queue

Describers with pod templates should have consistent output

Added a test to verify it.

Fixes #38698

Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)

Move node problem detector test into node e2e.

Move current NPD e2e test into node e2e.

In fact, current NPD e2e test is only a functionality test for NPD. It creates test NPD pod, sets test configuration, generates test logs and verifies test result.
It doesn't actually test the NPD really deployed in the cluster.

So it doesn't actually need to run in cluster e2e. Running it in node e2e will:
1) Make it easier to run the test.
2) Make it more light weight to introduce this as a pre/post submit test in NPD repo in the future.

Except this, I'm working on a cluster e2e to run some basic functionality test and benchmark test against the real NPD deployed in the cluster. Will send the PR later.

/cc @dchen1107 @kubernetes/node-problem-detector-reviewers

Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)

make edit respect --save-config

New behavior:

edit:
1) the flag is set to true: it will create or update the last-applied-config annotation
2) the flag is set to false or is unspecified: NOP

replace:
1) If saveConfig is true, create or update the annotation.
2) If saveConfig is false and the local config file doesn't have the annotation, we save the annotation from the live object if there is one (Try to not surprise the users).
3) If saveConfig is false and the local config file has the annotation, we use the annotation in the config file.
4) Same behavior for force replacing

fixes #40626

```release-note
stop kubectl edit from updating the last-applied-configuration annotation when --save-config is unspecified or false.
```

@pwittrock @liggitt 

This is a bug fix that prevented `edit` from being respected by `apply`

Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)

Add options to kubefed telling it to generate HTTP Basic and/or token credentials for the Federated API server

fixes #41265.

**Release notes**:
```release-note
Adds two options to kubefed, `-apiserver-enable-basic-auth` and `-apiserver-enable-token-auth`, which generate an HTTP Basic username/password and a token respectively for the Federated API server.
```

Automatic merge from submit-queue (batch tested with PRs 41984, 41682, 41924, 41928)

RC/RS: Fully Respect ControllerRef

**What this PR does / why we need it**:

This is part of the completion of the [ControllerRef](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md) proposal. It brings ReplicaSet and ReplicationController into full compliance with ControllerRef. See the individual commit messages for details.

**Which issue this PR fixes**:

Although RC/RS had partially implemented ControllerRef, they didn't use it to determine which controller to sync, or to update expectations. This could lead to instability or controllers getting stuck.

Ref: https://github.com/kubernetes/kubernetes/issues/24433

**Special notes for your reviewer**:

**Release note**:
```release-note
```
cc @erictune @kubernetes/sig-apps-pr-reviews

kubeadm: Add --cert-dir, --apiserver-cert-extra-sans, remove --api-external-dns-names and add the phase command for certs. Also use the CertificatesDir var everywhere instead of the HostPKIPath variable and fix some bugs in certs.go

 - The default is printed automatically
 - The string text did not match the actual default

Automatic merge from submit-queue

StorageClass should not print the namespace column

Fixes #40091

get fresh ports on startup failure for integration test

Automatic merge from submit-queue

Only set a groupresource on errors if a resource was specified

@deads2k this gets rid of the weird .meta.k8s.io error on unauthorized errors

Images field didn't include init containers. Use pod template output
consistently and ensure labels and annotations are in the right spot

Added a test to verify it.

Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)

Default E2E_ZONES to empty string.

Also print an error message and exit if host cluster context cannot be derived.

I don't think we should do any of this magic. This is a stop gap solution to get all our tests to stable state. I will actively get rid of all this defaulting magic once our main tests are stable.

cc @kubernetes/sig-federation-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)

Fix godep race in local builds

**What this PR does / why we need it**:
This fixes a small race condition when building a godep version other than the latest. As you're building godep once, and then building it again after checkout, this can cause inconsistency in the version selected for use. This fix tells the first `go get` to only download and not build. Then we build manually after checking out the target version.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 42128, 42064, 42253, 42309, 42322)

Fix panic on nil invalid field error

bug fix for validation panic

if a field.Invalid is constructed with a nil badvalue, the Error() method panics, since reflect.TypeOf() returns nil