Automatic merge from submit-queue (batch tested with PRs 64344, 64709, 64717, 63631, 58647). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support for enforcing read only host paths in PSPs. 

**What this PR does / why we need it**:

This PR adds support for the PSP to enforce that host paths are readonly. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57371
xref https://github.com/kubernetes/features/issues/5

**Special notes for your reviewer**:

**Release note**:

```release-note
PodSecurityPolicy now supports restricting hostPath volume mounts to be readOnly and under specific path prefixes
```

/cc @ericchiang @liggitt

Automatic merge from submit-queue (batch tested with PRs 64344, 64709, 64717, 63631, 58647). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add gpu cluster upgrade test.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
Currently running GPUMasterUpgrade test should pass with gpu nodes but running GPUClusterUpgrade test will run into https://github.com/kubernetes/kubernetes/issues/63506

**Release note**:

```release-note

```

Automatic merge from submit-queue (batch tested with PRs 64344, 64709, 64717, 63631, 58647). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

CoreDNS pull image from gcr.io

**What this PR does / why we need it**:
CoreDNS now pulls image from gcr.io
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 64344, 64709, 64717, 63631, 58647). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix panic caused by no cloudprovider in test

We should not panic when no cloudprovider is present

Fixes https://github.com/kubernetes/kubernetes/issues/64704

Also added a test to cover the panic.

/sig storage
/sig node

```release-note
None
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Teach Kubelet about Pod Ready++

Follow up PR of https://github.com/kubernetes/kubernetes/pull/62306 and https://github.com/kubernetes/kubernetes/pull/64057, **Only the last 3 commits are new.** Will rebase once the previous ones are merged.

ref: https://github.com/kubernetes/community/blob/master/keps/sig-network/0007-pod-ready%2B%2B.md


kind/feature
priority/important-soon
sig/network
sig/node

/assign @yujuhong


```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

API change for volume topology aware dynamic provisioning

**What this PR does / why we need it**:

Split PR https://github.com/kubernetes/kubernetes/pull/63193 for better review
part 2: API change

Previous: https://github.com/kubernetes/kubernetes/pull/63232
Next: https://github.com/kubernetes/kubernetes/pull/63193

**Which issue(s) this PR fixes** 
Feature: https://github.com/kubernetes/features/issues/561
Design: https://github.com/kubernetes/community/issues/2168

**Special notes for your reviewer**:
/sig storage
/sig scheduling
/assign @msau42 @jsafrane @thockin 


**Release note**:

```release-note
API change for volume topology aware dynamic provisioning
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add azuredisk PV size grow feature

**What this PR does / why we need it**:
According to kubernetes/features#284, add size grow feature for azure disk

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56463

**Special notes for your reviewer**:
 - This feature is ony for azure managed disk, and if that disk is already attached to a running VM, disk resize will fail as following:
```
$ kubectl describe pvc pvc-azuredisk
Events:
  Type     Reason              Age               From           Message
  ----     ------              ----              ----           -------
  Warning  VolumeResizeFailed  51s (x3 over 3m)  volume_expand  Error expanding volume "default/pvc-azuredisk" of plugin kubernetes.io/azure-disk : disk.DisksClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="OperationNotAllowed" Message="Cannot resize disk andy-mg1102-dynamic-pvc-d2d00dd9-6185-11e8-a6c3-000d3a0643a8 while it is attached to running VM /subscriptions/.../resourceGroups/.../providers/Microsoft.Compute/virtualMachines/k8s-agentpool-17607330-0."
```

**How to use this feature**
 - `kubectl edit pvc pvc-azuredisk` to change azuredisk PVC size from 6GB to 10GB
```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
...
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-disk
  creationTimestamp: 2018-05-27T08:13:23Z
  finalizers:
  - kubernetes.io/pvc-protection
  name: pvc-azuredisk
...
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 6Gi
  storageClassName: hdd
  volumeMode: Filesystem
  volumeName: pvc-d2d00dd9-6185-11e8-a6c3-000d3a0643a8
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 6Gi
  conditions:
  - lastProbeTime: null
    lastTransitionTime: 2018-05-27T08:14:34Z
    message: Waiting for user to (re-)start a pod to finish file system resize of
      volume on node.
    status: "True"
    type: FileSystemResizePending
  phase: Bound
```

 - After resized, `/mnt/disk` is still 6GB
```
$ kubectl exec -it nginx-azuredisk -- bash
# df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sdf        5.8G   15M  5.5G   1% /mnt/disk
...
```

 - After user run `sudo resize2fs /dev/sdf` in agent node, `/mnt/disk` becomes 10GB now:
```
$ kubectl exec -it nginx-azuredisk -- bash
# df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sdf        9.8G   16M  9.3G   1% /mnt/disk
...
```

**Release note**:

```
Add azuredisk size grow feature
```

/sig azure
/assign @feiskyer @karataliu @gnufied 
cc @khenidak

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Modify e2e tests to use priorityClass beta client version & switch priorityClass to beta

**What this PR does / why we need it**:
/cc @bsalamat @aveshagarwal @liggitt 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471

**Special notes for your reviewer**:
Once #57963 merges, this could go in.

**Release note**:

```release-note
Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix error handling in gc e2e test

**What this PR does / why we need it**:
Error messages were not being surfaced in log of GC e2e test, part of fixing #60463

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix test tag on dynamic config tests

The test accidentally got turned off when the NodeAlphaFeature tag was
added in #64125. This PR updates the tag to turn it back on.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding CSI driver registration with plugin watcher

Adding CSI driver registration bits.  The registration process will leverage driver-registrar side which will open the `registration` socket and will listen for pluginwatcher's GetInfo calls.
 
```release-note
Adding CSI driver registration code.
```
/sig sig-storage

Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Avoid deadlock in gc resync if available resources change during sync

retry GC sync if waiting for cache sync times out, without unpausing workers

viewing ignoring whitespace reveals the actual change:
https://github.com/kubernetes/kubernetes/pull/64235/files?w=1

xref https://github.com/kubernetes/kubernetes/pull/61057 https://github.com/kubernetes/kubernetes/pull/56446#discussion_r153381426

```release-note
fixes a potential deadlock in the garbage collection controller
```

Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix the handling of untagged images

**What this PR does / why we need it**:

This PR addresses the issue [#63354](https://github.com/kubernetes/kubernetes/issues/63354). If a container image is untagged, the default ":latest" tag will be appended to the image, preserving the semantics of the kubelet's handling and reporting the container images on a node.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

Fixes #63354

**Special notes for your reviewer**:

cc @resouer 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bind mount subpath with same read/write settings as underlying volume

**What this PR does / why we need it**:
https://github.com/kubernetes/kubernetes/pull/63045 broke two scenarios:
* If volumeMount path already exists in container image, container runtime will try to chown the volume
* In SELinux system, we will try to set SELinux labels when starting the container

This fix makes it so that the subpath bind mount will inherit the read/write settings of the underlying volume mount. It does this by using the "bind,remount" mount options when doing the bind mount.

The underlying volume mount is ro when the volumeSource.readOnly flag is set. This is for persistent volume types like PVC, GCE PD, NFS, etc.  When this is set, we won't try to configure SELinux labels.  Also in this mode, subpaths have to already exist in the volume, we cannot make new directories on a read only volume.

When volumeMount.readOnly is set, the container runtime is in charge of making the volume in the container readOnly, but the underlying volume mount on the host can be writable. This can be set for any volume type, and is permanently set for atomic volume types like configmaps, secrets.  In this case, SELinux labels will be applied before the container runtime makes the volume readOnly.  And subpaths don't have to exist.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #64120

**Special notes for your reviewer**:

**Release note**:

```release-note
Fixes issue for readOnly subpath mounts for SELinux systems and when the volume mountPath already existed in the container image.
```

Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add log and fs stats for Windows containers

**What this PR does / why we need it**:

Add log and fs stats for Windows containers.

Without this, kubelet will report errors continuously:

```
Unable to fetch container log stats for path \var\log\pods\2a70ed65-37ae-11e8-8730-000d3a14b1a0\echo: Du not supported for this build.
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #60180 #62047

**Special notes for your reviewer**:

**Release note**:

```release-note
Add log and fs stats for Windows containers
```

Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Clarify --hostname-override and --cloud-provider interaction

pairs with a PR to the website cloud provider page defining behavior for existing cloud providers: https://github.com/kubernetes/website/pull/8873

xref https://github.com/kubernetes/kubernetes/issues/64659 https://github.com/kubernetes/kubernetes/issues/62600 https://github.com/kubernetes/kubernetes/issues/61774 https://github.com/kubernetes/kubernetes/issues/54482

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Replace glog.Info{f,ln} with fmt.Print{f,ln}

Please see commit message.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>

**What this PR does / why we need it**:
This PR makes kubeadm's logging output consistent for the regular case and offers verbosity with glog.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubernetes/kubeadm#852

**Special notes for your reviewer**:
Unfortunately there is one call site in a dependency that uses a bare glog.Infof which we can't change without a bigger effort.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

client-go: promote exec plugin support to beta

/sig auth
/kind feature

Adds a "v1beta1" API group for client authentication. Because of a lack of usage with the input parameters, these have been dropped for the beta. Would like to re-introduce them as users feel they require them.

updates https://github.com/kubernetes/kubernetes/issues/61796

```release-note
client-go: credential exec plugins have been promoted to beta
```

Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "Remove rescheduler and corresponding tests from master"

Reverts kubernetes/kubernetes#64364

After discussing with @bsalamat on how DS controllers(ref: https://github.com/kubernetes/kubernetes/pull/63223#discussion_r192277527) cannot create pods if the cluster is at capacity and they have to rely on rescheduler for making some space, we thought it is better to 

- Bring rescheduler back.
- Make rescheduler priority aware.
- If cluster is full and if **only** DS controller is not able to create pods, let rescheduler be run and let it evict some pods which have less priority.
- The DS controller pods will be scheduled now.

So, I am reverting this PR now. Step 2, 3 above are going to be in rescheduler.

/cc @bsalamat @aveshagarwal @k82cn 

Please let me know your thoughts on this. 

```release-note
Revert #64364 to resurrect rescheduler. More info https://github.com/kubernetes/kubernetes/issues/64725 :)
```

Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add Makefile target to push fat manifest for multi-arch images
Signed-off-by: Nitesh Konkar <niteshkonkar@in.ibm.com>



**What this PR does / why we need it**:
This PR adds support to push fat manifest for multi-arch images. We need this
so that we can seamlessly pull and run the test images on multiple platforms. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Introduce priority class in the resource quota

**What this PR does / why we need it**:
Implements https://github.com/kubernetes/community/pull/933
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #48648 

**Special notes for your reviewer**:
Test Cases are still to be covered. Opening this PR to make discussion convenient with code references.
Will update test cases only after design PR has got merged.

**Release note**:

```release-note
Ability to quota resources by priority
```
/kind feature
/priority important-soon
/sig scheduling
/sig node
/cc @resouer @derekwaynecarr @sjenning @bsalamat @timstclair @aveshagarwal @ravisantoshgudimetla

Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add parametrizable priority function mapping requested/capacity ratio to priority

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add PodSecurityPolicy information to audit logs

Depends on: https://github.com/kubernetes/kubernetes/pull/58806
Fix #56209

**Release note**:
```release-note
PodSecurityPolicy admission information is added to audit logs
```

Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add netd as an addon for GCP

**What this PR does / why we need it**:
Add netd as an addon for GKE.
The PR will add setup functions and var to help deploy netd daemon on GKE.
Please checkout more detail for netd at https://github.com/GoogleCloudPlatform/netd

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Create CoreDNS and kube-dns folders

**What this PR does / why we need it**:
Separate the CoreDNS and kube-dns manifests by creating their own folders (dns/coredns and dns/kube-dns) 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #61435 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
cc @MrHohn

We should not panic when no cloudprovider is present