Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add stub device plugin for conformance e2e test

**What this PR does / why we need it**:

Add stub device plugin for conformance e2e test
- extend [device_plugin_stub](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/cm/deviceplugin/device_plugin_stub.go) to support e2e test
- add test suite with this device-plugin-stub
- simulate more use cases by deploying some pods to request these resources

**Which issue this PR fixes**:

fixes #52861

**Special notes for your reviewer**:

@vishh @jiayingz PTAL.

**Release note**:

```release-note
None
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

client-go: remove open-api import from types

This builds off of https://github.com/kubernetes/kube-openapi/pull/21 and removes the following imports from `k8s.io/client-go/kubernetes`:

```
github.com/PuerkitoBio/purell
github.com/PuerkitoBio/urlesc
github.com/emicklei/go-restful
github.com/emicklei/go-restful/log
github.com/go-openapi/jsonpointer
github.com/go-openapi/jsonreference
github.com/go-openapi/spec
github.com/go-openapi/swag
github.com/mailru/easyjson/buffer
github.com/mailru/easyjson/jlexer
github.com/mailru/easyjson/jwriter
golang.org/x/text/cases
golang.org/x/text/internal
golang.org/x/text/internal/tag
golang.org/x/text/language
golang.org/x/text/runes
golang.org/x/text/secure/precis
golang.org/x/text/width
k8s.io/kube-openapi/pkg/common
```

/assign @sttts 
/assign @mbohlool 

cc @kubernetes/sig-api-machinery-pr-reviews 


```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[Alpha: DynamicKubeletConfig] Double check before setKubeletConfiguration

**What this PR does / why we need it**:

Double check the `newCfg` is not equal to the `oldCfg` before we call `setKubeletConfiguration(newCfg)` in `tempSetCurrentKubeletConfig()`.

**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubernetes/issues/57701

**Special notes for your reviewer**:

/area kubelet
/sig node
/assign @mtaufen 
/cc @vishh @jiayingz @derekwaynecarr @dchen1107 @liggitt 
PTAL, Thanks!

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove dead code in pkg/api

**What this PR does / why we need it**:
remove dead code
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add admission into RecommendedOptions

**What this PR does / why we need it**:

make admission part of the RecommendedOptions

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56627

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix CHANGELOG urls for release 1.9.1

Something went wrong in running anago for 1.9.1 release that resulted in private repo's URLs but public repo's SHA256 for all files. I've checked several SHA256s and all of them were right after pointing to the public files. This is a manual fix for changelog. We need to figure out what went wrong with the release script too. @david-mcmahon 

fixes #58120

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixing logs for cri stats

**What this PR does / why we need it**:
This is a follow up PR of https://github.com/kubernetes/kubernetes/pull/55810 to clean up couple of logs in cri stats code.
Signed-off-by: abhi <abhi@docker.com>

cc @Random-Liu

Signed-off-by: abhi <abhi@docker.com>

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

git: Use VolumeHost.GetExec() to execute stuff in volume plugins

This prepares volume plugins to run things in containers instead of running
them on the host.

**Special notes for your reviewer**:

* instead of `cmd.SetDir(<dir>); cmd.Exec("git <command>")`, we do `cmd.Exec("git -C <dir> <command>")` - mounter.Exec does not have SetDir()
* there are lot of changes in the tests because a different exec interface is used.

@kubernetes/sig-storage-pr-reviews 

```release-note
gitRepo volumes in pods require git 1.8.5 or later
```

/assign @rootfs

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "no need delete endpoint explicitly in endpoint controller"

Reverts kubernetes/kubernetes#57747

For: 
1.  `checkLeftoverEndpoints` does not work any more.
2. service delete auto trigger endpoint delete in apiserver, if service is successfully deleted but fail on endpoint delete. Then this endpoint will be garbage resource.

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix TestCadvisorListPodStats failure under mac/darwin

**What this PR does / why we need it**:
GetPodCgroupNameSuffix is not really implemented under darwin
(or windows for that matter). So let's just skip over the check
for CPU and Memory if that is not set.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57636

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix bug:Kubelet failure to umount  mount points

What this PR does / why we need it:
Fix bug#41141:Kubelet failure to umount glusterfs mount points
kubelet failure to umount mount moints in these cases:
1. glusterfs volume : transport endpoint is not connected
2. nfs volume : stale NFS file handle
this PR will fix it

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
fixes #41141
Special notes for your reviewer:
NONE
Release note:
NONE

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Wrap azure client calls

**What this PR does / why we need it**:

This is a clean up for azure client calls. It adds wrappers over azure clients and moves verbose logs and rate limiter inside.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/cc @cosmincojocar  @andyzhangx

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix cinder detach problems

**What this PR does / why we need it**: We have currently huge problems in cinder volume detach. This PR tries to fix these issues.

**Which issue(s) this PR fixes**:
Fixes #50004 
Fixes #57497

**Special notes for your reviewer**: 
**Release note**:

```release-note
openstack cinder detach problem is fixed if nova is shutdowned
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

client-go: remove import of github.com/gregjones/httpcache

Moves NewCacheRoundTripper from `k8s.io/client-go/transport` to its own package. This prevents Kubernetes clients from requiring its dependencies.

This change removes the following transitive imports from `k8s.io/client-go/kubernetes`

```
github.com/google/btree
github.com/gregjones/httpcache
github.com/gregjones/httpcache/diskcache
github.com/peterbourgon/diskv
```


```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add zouyee as a reviewer for the cluster/centos directory.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

integration: add retries to node authorizer tests

Fixes https://github.com/kubernetes/kubernetes/issues/57769

cc @kubernetes/sig-auth-pr-reviews 

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Print the full path of Kubeconfig files.

**What this PR does / why we need it**:
kubeadm init:
```
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
```

It'd be good to print the path of the KubeConfig files in case users want to view the content of these files.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubeadm/issues/656

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix quota controller worker deadlock

The resource quota controller worker pool can deadlock when:

* Worker goroutines are idle waiting for work from queues
* The Sync() method detects discovery updates to apply

The problem is workers acquire a read lock while idle, making write lock
acquisition dependent upon the presence of work in the queues.

The Sync() method blocks on a pending write lock acquisition and won't unblock
until every existing worker processes one item from their queue and releases
their read lock. While the Sync() method's lock is pending, all new read lock
acquisitions will block; if a worker does process work and release its lock, it
will then become blocked on a read lock acquisition; they become blocked on
Sync(). This can easily deadlock all the workers processing from one queue while
any workers on the other queue remain blocked waiting for work.

Fix the deadlock by refactoring workers to acquire a read lock *after* work is
popped from the queue. This allows writers to get locks while workers are idle,
while preserving the worker pause semantics necessary to allow safe sync.

```release-note
Fixes an infrequent problem causing the resource quota controller to become stuck in clusters with low ResourceQuota churn, potentially preventing quota from being recalculated until the controller is restarted or until bursts of diverse quota activity unstick the controller.
```

/cc @kubernetes/sig-api-machinery-bugs

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix cadvisor flag registration for cross build

Fixed cross build by only importing linux-only cadvisor packages on linux (unconditional import added in https://github.com/kubernetes/kubernetes/pull/57613)

Fixes https://github.com/kubernetes/kubernetes/issues/58106

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

added fluentd detect-exceptions plugin to fluentd-es-image for multiline logging

**What this PR does / why we need it**:
This PR adds proper multiline logging (java stacktraces for example) to fluentd-es docker image by adding the fluentd detect-exceptions plugin.

Plugin: https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions

Its also used in GCP and it also works outside of Kubernetes, so it should be safe to use it here too.


**Release note**:
```release-note
fluentd-es addon: multiline stacktraces are now grouped into one entry automatically
```

The resource quota controller worker pool can deadlock when:

* Worker goroutines are idle waiting for work from queues
* The Sync() method detects discovery updates to apply

The problem is workers acquire a read lock while idle, making write lock
acquisition dependent upon the presence of work in the queues.

The Sync() method blocks on a pending write lock acquisition and won't unblock
until every existing worker processes one item from their queue and releases
their read lock. While the Sync() method's lock is pending, all new read lock
acquisitions will block; if a worker does process work and release its lock, it
will then become blocked on a read lock acquisition; they become blocked on
Sync(). This can easily deadlock all the workers processing from one queue while
any workers on the other queue remain blocked waiting for work.

Fix the deadlock by refactoring workers to acquire a read lock *after* work is
popped from the queue. This allows writers to get locks while workers are idle,
while preserving the worker pause semantics necessary to allow safe sync.

Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block volumes Support: iSCSI plugin update

**What this PR does / why we need it**:

Add interface changes to iSCSI volume plugin to enable block volumes support feature.

**Which issue this PR fixes**: 
Based on this proposal (kubernetes/community#805 & kubernetes/community#1265) and this feature issue: kubernetes/features#351

**Special notes for your reviewer**:

This PR temporarily includes following changes except iSCSI plugin change for reviewing purpose.
These changes will be removed from the PR once they are merged.
- (#50457) API Change 
- (#51494) Container runtime interface change, volumemanager changes, operationexecutor changes

There are another PRs related to this functionality.
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
(#55112) Block volume: Command line printer update
Plugins
(#51493) Block volumes Support: FC plugin update
(#54752) Block volumes Support: iSCSI plugin update

**Release note**:

```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

GCE: Use existing subnetwork of ILB forwarding rule

Fixes #57860

**Test Cases**:
Clusters using auto network with existence of a manual subnet in same region.
- [x] Upgrade 1.7 cluster with existing ILBs to latest. Confirm existing ILBs still are synced.
  Version 1.7 does not attempt to fill in the subnetwork, so the forwarding rule was created with the correct subnetwork. 
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using wrong subnet) still are synced. 
- [x]  Latest version creates ILBs using the correct subnet.


Clusters with manual subnets have always and will continue to use the subnet specified in gce.conf.
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using manual subnet) still are synced. 


Clusters with legacy networks have always and will continue to use an empty subnet. 
- [x] Upgrade 1.8 cluster with existing ILBs to latest. Confirm existing ILBs (using legacy network) still are synced. 


**Release note**:
```release-note
GCE: Allows existing internal load balancers to continue using an outdated subnetwork 
```

Automatic merge from submit-queue (batch tested with PRs 54230, 58100, 57861, 54752). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add a feature flag for sharing process namespace between containers in a pod

**What this PR does / why we need it**:
This adds a feature flag for the feature described in kubernetes/features#495.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/assign @dchen1107 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Allow Ceph server some time to start

Ceph server needs to create our "foo" volume on startup. It keeps the image small, however it makes the server container start slow.

Add sleep before the server is usable. Without this PR, all pods that use Ceph fail to start for couple of seconds with cryptic "image foo not found" error and it clutters logs and pod logs and makes it harder to spot real errors.

```release-note
NONE
```

/sig storage
/sig testing
/assign @jeffvance @copejon

Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

ignore images in used by running containers when GC

**What this PR does / why we need it**:
Let kubelet not attempt to remove images being used by running containers.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57006

**Special notes for your reviewer**:
@kubernetes/sig-node-pr-reviews 

**Release note**:

```release-note
ignore images in used by running containers when GC
```

Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove deprecated photon controller kube-up

This kube-up hasn't been maintained for over a year and has been deprecated for > 6 months.

see #49213 

```release-note
Remove deprecated and unmaintained photon-controller kube-up.sh. 
```

Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Mark kubelet PID namespace flag as deprecated

**What this PR does / why we need it**:

The `--docker-disable-shared-pid` flag will be removed once per-pod configurable process namespace sharing becomes available. Mark it deprecated to notify cluster admins of the upcoming change.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
WIP #41938

**Special notes for your reviewer**:
/assign @yujuhong 

**Release note**:

```release-note
The kubelet flag to run docker containers with a process namespace that is shared between all containers in a pod is now deprecated and will be replaced by a new field in `v1.Pod` that configures this behavior.
```

Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

manually handle encoding and decoding in the scale client

@p0lyn0mial I think this resolves the problems you're seeing in https://github.com/kubernetes/kubernetes/pull/57843 by manually controlling the encode and decode in the client itself.



/assign @DirectXMan12

Automatic merge from submit-queue (batch tested with PRs 57823, 58091, 58093, 58096, 57020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix rbd ConstructVolumeSpec bug

**What this PR does / why we need it**:

Fix rbd `ConstructVolumeSpec` bug as its global device mount path is not canonical. Full details are in #57744 .

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57744 

**Release note**:

```release-note
NONE
```

/sig storage
/kind bug

Automatic merge from submit-queue (batch tested with PRs 58025, 57112, 57879, 57571, 58062). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fixed some bad url

**What this PR does / why we need it**:

fixed some bad url

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58025, 57112, 57879, 57571, 58062). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Treat staging repos as authoritative source for all files

Add CONTRIBUTING.md files to remind people not to PR directly into the published repos.

/hold
I believe this requires removing these files from the published repos if any copies exist there

ref: kubernetes/kubernetes#57559

Automatic merge from submit-queue (batch tested with PRs 58025, 57112, 57879, 57571, 58062). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Code generation for GCE compute interface

Use code generation to "write" most of the GCE cloud provider library. This enables the following:

- Consistent interfaces, including handling of the different API versions (GA, alpha, beta)
- Efficient implementation of cross cutting features such as metrics, logging, tracing etc. Adding such features has in the past been a tedious and error prone endeavor. 
- High fidelity mocks for all of the compute API. What this means is that most of our controller logic can be tested as unit tests in a consistent way without creating individual mocks by hand.

```release-note
NONE
```