Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)

test/e2e/auth: fix audit log test format parsing

Fixes https://github.com/kubernetes/kubernetes/issues/51556

```release-note
NONE
```

cc @CaoShuFeng

Still need to figure out how to run this test locally.

Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)

fixing a typo in staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go

**What this PR does / why we need it**:
Fix a typo in `staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

`NONE`

Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)

GCE: Reserve address for ILBs during sync

**What this PR does / why we need it**:
This PR adds the ability for the service controller to hold the ILB's IP during sync which may delete/recreate the forwarding rule.

Fixes: #47531

**Release note**:
```release-note
GCE: Internal load balancer IPs are now reserved during service sync to prevent losing the address to another service.
```

Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)

[Scheduler] Fix typo in info message
Signed-off-by: sakeven <jc5930@sina.cn>



**What this PR does / why we need it**:

Missed a `'` in info message.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
NONE
```

Automatic merge from submit-queue

Use json-iterator instead of ugorji for JSON.

@smarterclayton @wojtek-t

Fixes #36120
xref #18762

```release-note
Switch JSON marshal/unmarshal to json-iterator library.  Performance should be close to previous with no generated code.
```

Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)

Separate feature gates for dynamic kubelet config vs loading from a file

This makes it so these two features can be turned on independently, rather than bundling both under dynamic kubelet config.

fixes: #51664

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)

Add some initial shell parsing tests.

These just test to see if there is a bash syntax error in these shell
libraries.

For #51642

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)

kubectl: Remove ending punctuation from error strings

**What this PR does / why we need it**:

`golint` emits 2 warnings of type:

`error strings should not end with punctuation`

Remove punctuation from end of error strings.



**Release note**:

```release-note
NONE
```
/sig cli
/kind cleanup

Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)

CPU manager wiring and `none` policy

Blocker for CPU manager #49186 (4 of 6)

* Previous PR in this series: #51140
* Next PR in this series: #51180

cc @balajismaniam @derekwaynecarr @sjenning 

**Release note**:

```release-note
NONE
```

TODO:
- [X] In-memory CPU manager state
- [x] Kubelet config value
- [x] Feature gate
- [X] None policy
- [X] Unit tests
- [X] CPU manager instantiation
- [x] Calls into CPU manager from Kubelet container runtime

Automatic merge from submit-queue (batch tested with PRs 49971, 51357, 51616, 51649, 51372)

add information for subresource kind determination

xref https://github.com/kubernetes/kubernetes/issues/38810 https://github.com/kubernetes/kubernetes/issues/38756

Polymorphic subresources usually have different groupVersions for their discovery kinds than their "native" groupVersions.  Even though the APIResourceList shows the kind properly, it does not reflect the group or version of that kind, which makes it impossible to unambiguously determine if the subresource matches you and it is impossible to determine how to serialize your data.  See HPA controller.

This adds an optional Group and Version to the discovery doc, which can be used to communicate the "native" groupversion of an endpoint.  Doing this does not preclude fancier contenttype negotiation in the future and doesn't prevent future expansion from indicating equivalent types, but it does make it possible to solve the problem we have today or polymorphic categorization.

@kubernetes/sig-api-machinery-misc @smarterclayton 
@cheftako since @lavalamp is out.

```release-note
Adds optional group and version information to the discovery interface, so that if an endpoint uses non-default values, the proper value of "kind" can be determined. Scale is a common example.
```

Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

GCE metadata proxy blocks instance identity & recursive calls, & excludes port from redirects

**What this PR does / why we need it**: Metadata proxy blocks [instance identity](https://cloud.google.com/compute/docs/instances/verifying-instance-identity) & [recursive](https://cloud.google.com/compute/docs/storing-retrieving-metadata#aggcontents) calls, and no longer includes port in redirects (it was serving redirects to `http://metadata.google.internal:988`, which doesn't resolve.  Ref #8867.

**Special notes for your reviewer**: Container is defined https://github.com/kubernetes/contrib/tree/master/metadata-proxy; I plan to send a separate PR to remove the `nginx.conf` directly in the container to reduce confusion.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

Ensure that DaemonSet respects termination

**What this PR does / why we need it**:
#43077 correctly prevents the DaemonSet controller from adopting deleted Pods, but, as pointed out in #50477, the controller now has no sensitivity to the termination lifecycle (i.e TerminationGracePeriodSeconds) of the Pods it creates. This PR attempts to balance the two. DaemonSet controller will now consider deleted Pods owned by a DaemonSet during creation, but it will not consider deleted Pods as targets for adoption.

fixes #50477

```release-note
#43077 introduced a condition where DaemonSet controller did not respect the TerminationGracePeriodSeconds of the Pods it created. This is now corrected.
```

Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

Fix pod local ephemeral storage usage calculation

We use podDiskUsage to calculate pod local ephemeral storage which is not correct, because podDiskUsage also contains HostPath volume  which is considered as persistent storage
This pr fixes it
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51489

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

/assign @jingxu97  @vishh 
cc @ddysher

Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

Fix printISCSIVolumeSource to show kubectl describe properly

**What this PR does / why we need it**:

After merging #51189, 'kubectl describe' doesn't show persistent volume properly if the volume is iSCSI.
This PR fixes the problem.


**Which issue this PR fixes** : Fixes #51635

**Special notes for your reviewer**:

Result without InitiatorName.
```
% k describe pv pv0001
Name:		pv0001
Labels:		<none>
Annotations:	volume.beta.kubernetes.io/storage-class=slow
StorageClass:	slow
Status:		Available
Claim:		
Reclaim Policy:	Recycle
Access Modes:	RWO
Capacity:	1Gi
Message:	
Source:
    Type:		ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
    TargetPortal:	192.168.122.85:3260
    IQN:		iqn.2017-05.com.example:rhel7
    Lun:		0
    ISCSIInterface	default
    FSType:		ext4
    ReadOnly:		true
    Portals:		[192.168.122.7:3260]
    DiscoveryCHAPAuth:	false
    SessionCHAPAuth:	false
    SecretRef:		<nil>
    InitiatorName:	<none>
Events:			<none>
```
Result with InitiatorName.
```
% k describe pv pv0001
Name:		pv0001
Labels:		<none>
Annotations:	volume.beta.kubernetes.io/storage-class=slow
StorageClass:	slow
Status:		Available
Claim:		
Reclaim Policy:	Recycle
Access Modes:	RWO
Capacity:	1Gi
Message:	
Source:
    Type:		ISCSI (an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod)
    TargetPortal:	192.168.122.85:3260
    IQN:		iqn.2017-05.com.example:rhel7
    Lun:		0
    ISCSIInterface	default
    FSType:		ext4
    ReadOnly:		true
    Portals:		[192.168.122.7:3260]
    DiscoveryCHAPAuth:	false
    SessionCHAPAuth:	false
    SecretRef:		<nil>
    InitiatorName:	iqn.1994-05.com.redhat:185ce16b55ad
Events:			<none>
```

@rootfs @humblec @jsafrane 

**Release note**:
```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51628, 51637, 51490, 51279, 51302)

Fix iSCSI WaitForAttach not mounting a volume

WaitForAttach failed consistently with this error:
```
Heuristic determination of mount point failed:stat /var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/10.128.0.3:3260-iqn.2003-01.org.linux-iscsi.f21.x8664:sn.4b0aae584f7c-lun-0: no such file or directory
```

We should ignore "no such file or directory" eror, the directory is created just few lines below.

Fixes: #51625



**Release note**:

```release-note
NONE
```

/sig storage
/assign @mtanino

Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)

audit policy: support subresources and resource names

Updates #48561

	policy:
	- level: Metadata
	  resources:
	  - group: ""
	    resources ["pods/logs"]
	- level: None
	  resources:
	  - group: ""
	    resources: ["configmaps"]
	    resourceNames: ["controller-leader"]

The top level resource no longer matches the subresource. For example "pods"
no longer matches requests to the logs subresource on pods.

```release-note
Audit policy supports matching subresources and resource names, but the top level resource no longer matches the subresouce. For example "pods" no longer matches requests to the logs subresource of pods. Use "pods/logs" to match subresources.
```

/cc @sttts @soltysh @crassirostris @ihmccreery @timstclair

Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)

Add a persistent volume label controller to the cloud-controller-manager

Part of https://github.com/kubernetes/features/issues/88

Outstanding concerns needing input:
- [x] Why 5 threads for controller processing?
- [x] Remove direct linkage to aws/gce cloud providers [#51629]
- [x] Modify shared informers to allow added event handlers ability to include uninitialized objects/using unshared informer #48893
- [x] Use cache.MetaNamespaceKeyFunc in event handler?

I'm willing to work on addressing the removal of the direct linkage to aws/gce after this PR gets in.

Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)

Task 1: Tainted node by condition.

**What this PR does / why we need it**:
Tainted node by condition for MemoryPressure, OutOfDisk and so on.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: part of #42001 

**Release note**:
```release-note
Tainted nodes by conditions as following:
  * 'node.kubernetes.io/network-unavailable=:NoSchedule' if NetworkUnavailable is true
  * 'node.kubernetes.io/disk-pressure=:NoSchedule' if DiskPressure is true
  * 'node.kubernetes.io/memory-pressure=:NoSchedule' if MemoryPressure is true
  * 'node.kubernetes.io/out-of-disk=:NoSchedule' if OutOfDisk is true
```

Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)

update scheduler to return structured errors instead of process exit

The scheduler Run method returns an error that is properly handled at higher levels.  Instead of existing the process, we should return the error and handle it at higher level logic to allow testing of error conditions and composition of commands.  The changes are relatively minor.

@sjenning @aveshagarwal

Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)

rbd: default image format to v2 instead of deprecated v1

**What this PR does / why we need it**:
Image format v1 has been deprecated since the Infernalis release of
Ceph over two years ago.

**Release note**:
```StorageClass Ceph RBD now defaults to using the v2 image format ```

Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

Add PVCRef to VolumeStats

**What this PR does / why we need it**:
For pod volumes that reference a PVC, add a PVCRef to the corresponding
volume stat. This allows metrics to be indexed/queried by PVC name
which is more user-friendly than Pod reference

**Which issue this PR fixes** : [#363](https://github.com/kubernetes/features/issues/363)

**Special notes for your reviewer**:

**Release note**:
```
`VolumeStats` reported by the kubelet stats summary API 
(http://<node>:10255/stats/summary) now include a PVCRef
field describing the PVC referenced by the volume (if any). 
```

Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

Add unit test for UploadConfig in Kubeadm

**What this PR does / why we need it**:
This is PR add unit test for UploadConfig refactoring which saved the master configuration used for init into ConfigMap for later use
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #https://github.com/kubernetes/kubeadm/issues/379

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

Removes redundant prefix in cluster-lifecycle e2e test names

**What this PR does / why we need it**:
Removes redundant prefix in cluster-lifecycle e2e test names

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Umbrella issue #49161
xref: #50054

**Special notes for your reviewer**:
/cc @jbeda 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

fix typo about volumes

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51513, 51515, 50570, 51482, 51448)

implementation of GetZoneByProviderID and GetZoneByNodeName for AWS

This a part of the #50926 effort

cc @luxas 

**Release note**:

```release-note
None
```

Automatic merge from submit-queue (batch tested with PRs 50719, 51216, 50212, 51408, 51381)

Surface reasonable error when connection closed

Try to detect connection closure when API server closed the connection due to timeout.
Surface reasonable error when connection closed.

Further improvement may be retrying when detect connection closure

related to #51353

```release-note
Surface reasonable error when client detects connection closed.
```

/assign @mml @caesarxuchao

Automatic merge from submit-queue (batch tested with PRs 50719, 51216, 50212, 51408, 51381)

Use constants instead of magic string for runtime names

**What this PR does / why we need it**:

Use constants instead of magic string for runtime names.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51678

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 50719, 51216, 50212, 51408, 51381)

Allow PSP's to specify a whitelist of allowed paths for host volume

**What this PR does / why we need it**:

Reverts the revert for the allowed host path feature that was moved from v1.7 to v1.8. This PR also changes the api as discussed in https://github.com/kubernetes/kubernetes/pull/47811.

Original pr: https://github.com/kubernetes/kubernetes/pull/43946
revert: https://github.com/kubernetes/kubernetes/pull/47851

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #29326

**Special notes for your reviewer**:

cc @liggitt @ericchiang @php-coder 

It seems the api build process has changed. Hopefully I did it right 👼 .

**Release note**:

```release-note
Allow PSP's to specify a whitelist of allowed paths for host volume
```

Automatic merge from submit-queue (batch tested with PRs 50719, 51216, 50212, 51408, 51381)

Fix prefixing bug in import verifier

In order to check if an import is of an allowed tree, we need to check
that the import is either literally to the base of the tree or that the
import is below the tree (the import, suffixed with `/`, should be a
prefix) instead of checking simply that the import is a prefix of the
allowed tree, as that causes issues with packages that are prefixes of
each other, like `k8s.io/api` and `k8s.io/apimachinery`.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

```release-note
NONE
```

Fixes https://github.com/kubernetes/kubernetes/issues/51212
/assign @sttts @deads2k 

This will be broken right now hopefully you'll be able to build on top to fix it @sttts

Automatic merge from submit-queue (batch tested with PRs 50719, 51216, 50212, 51408, 51381)

Make selector immutable for v1beta2 deployment, replicaset and daemonset prior update

**What this PR does / why we need it**:
This PR ensures controller selector is immutable for deployment and replicaset prior update by ignoring any change to `Spec`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50808

**Special notes for your reviewer**:
This will be a breaking change.

**Release note**:

```release-note
For Deployment, ReplicaSet, and DaemonSet, selectors are now immutable when updating via the new `apps/v1beta2` API. For backward compatibility, selectors can still be changed when updating via `apps/v1beta1` or `extensions/v1beta1`.
```

Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)

Kubectl to use http caching to cache openapi responses from the server

**What this PR does / why we need it**: 

This PR is trying to address the problems raised in #50254 

> * uses a disk-based cache that is not safe between processes (does not use atomic fs operations)
> * writes get/list responses to disk that should not be cached (like kubectl get secrets)
> * is vulnerable to partially written cache responses being used as responses to future requests
> * breaks uses of the client transport that make use of websockets
> * defaults to enabling the cache for any client builder using RecommendedConfigOverrideFlags or DefaultClientConfig which affects more components than just kubectl

All of these points are addressed by this pull-request:
1. It now uses atomic fs operations
2. Doesn't cache by default, only if requested by the client (and it's only done by openapi client)
3. Fixed because of atomic fs operations
4. Found the reason for the bug: Cache wrapper couldn't be unwrapped. I implemented the `WrappedRoundTripper` interface.
5. Since 2. is fixed, I think that should be fine

@smarterclayton @liggitt 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50254

**Special notes for your reviewer**:

**Release note**:

```release-note
Allows kubectl to use http caching mechanism for the OpenAPI schema. The cache directory can be configured through `--cache-dir` command line flag to kubectl. If set to empty string, caching will be disabled.
```

Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)

Use 'Infof' instead of 'Errorf' for a debug log

Outputing error log for a debug is confused.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)

make get pod log with follow option as CONNECT verb

**What this PR does / why we need it**:
Don't make the get log with follow option request mix with GET pods request. Make it reported as a WATCH pod log request.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes #49998 

```release-note
Pod log attempts are now reported in apiserver prometheus metrics with verb `CONNECT` since they can run for very long periods of time.
```

Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)

getHashEquivalencePod should return if equivalence pod is found
Signed-off-by: sakeven <jc5930@sina.cn>



**What this PR does / why we need it**:

getHashEquivalencePod should return if equivalence pod is found, rather than simply check equivalenceHash equals to 0 later.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)

Add KUBE_APISERVER_REQUEST_TIMEOUT_SEC env var.

Cluster startup support for the flag added by #51415.  I won't merge until that PR merges.

Bug: #51355

cc @jpbetz

Automatic merge from submit-queue (batch tested with PRs 51707, 51662, 51723, 50163, 51633)

update GC controller to wait until controllers have been initialized …

fixes #51013

Alternative to https://github.com/kubernetes/kubernetes/pull/51492 which keeps those few controllers (only one) from starting the informers early.

Automatic merge from submit-queue (batch tested with PRs 51707, 51662, 51723, 50163, 51633)

Change SizeLimit to a pointer

This PR fixes issue #50121

```release-note
The `emptyDir.sizeLimit` field is now correctly omitted from API requests and responses when unset.
```