Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)

Use multiple clients in the density test

Fix #47954

Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)

Add e2e tests for CA scale up when pending pod requests volume

Test verifying pending pods with PVC don't interfere with scale up, issue: kubernetes/autoscaler#22

Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)

Bumped Heapster to v1.4.0

``` release-note
Bumped Heapster to v1.4.0.
More details about the release https://github.com/kubernetes/heapster/releases/tag/v1.4.0
```

follow up #47961
The release candidate `v1.4.0-beta.0` turned out to be stable.

Automatic merge from submit-queue (batch tested with PRs 48004, 48205, 48130, 48207)

Do not set CNI in cases where there is a private master and network policy provider is set.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
In GCE and in a "private master" setup, do not set the network-plugin provider to CNI by default if a network policy provider is given.
```

Automatic merge from submit-queue (batch tested with PRs 48192, 48182)

Add generic NoSchedule toleration to fluentd in gcp config as a quick…

…-fix for #44445

Fix bug cluster-subnet logic

Automatic merge from submit-queue

kubeadm: Start using Tolerations in yaml code again and unit-test

**What this PR does / why we need it**:

- Earlier there was a problem with decoding Tolerations from yaml. Seems to be fixed now.
- Added an unit test to catch such a failure if that regression ever happens again

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

Targets v1.8

**Release note**:

```release-note
NONE
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @timothysc

Automatic merge from submit-queue

kubeadm: Expose only the cluster-info ConfigMap in the kube-public ns

**What this PR does / why we need it**:

Noticed a bug; we should only expose the `cluster-info` ConfigMap.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes: https://github.com/kubernetes/kubeadm/issues/320

**Special notes for your reviewer**:

Cherrypick-candidate for v1.8 cc @dchen1107 
Not blocking the release though...

**Release note**:

```release-note
NONE
```
@jbeda @pipejakob @timothysc @kubernetes/sig-cluster-lifecycle-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 48123, 48079)

kubectl/cmd: many small refactors

#### Improve Code Quality in `pkg/kubectl/cmd`

___

 * [x] Rename variables and functions to match Go convention. For example, UsageError --> UsageErrorf.
 * [x] Remove redundant or unreachable code.
 * [x] Simplify some utility functions (no functionality changes).
 * [x] Fix hanging 'if { return } else { return }' constructs.
 * [x] Fix several incorrect printf verbs.

These changes were extracted from the refactoring and flag parity work I'm doing in kubectl.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48123, 48079)

[Kubelet] Fix race condition in container manager

**What this PR does / why we need it**:

This fixes a race condition where the container manager capacity map was being updated without synchronization.  It moves the storage capacity detection to kubelet initialization, which happens serially in one thread.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48045

**Release note**:

```release-note
Fixes kubelet race condition in container manager.
```

Automatic merge from submit-queue

Changes node e2e tests to use the new Ubuntu image

ref: https://github.com/kubernetes/kubernetes/issues/46891

`ubuntu-docker10` and `ubuntu-docker12` images are deprecated in favor of the new one.

**Release note**:
```
None
```
/sig node
/area node-e2e
/assign @dchen1107

Automatic merge from submit-queue (batch tested with PRs 48118, 48159)

Bump GCE ContainerVM to container-vm-v20170627

Remove the built-in kubelet (finally), pick up security fixes.

```release-note
Bump GCE ContainerVM to container-vm-v20170627
```

Not really associated issue: #48163

Automatic merge from submit-queue (batch tested with PRs 48118, 48159)

Ensures node becomes schedulable at the end of tests that delete nodes

**What this PR does / why we need it**: Further fixes the flakiness of "Pod Disk should be able to detach from a node which was deleted". When a node becomes ready but not schedulable, it was not included in the final node count.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48008 

**Special notes for your reviewer**: Updated a similar test, "Pod Disk should be able to detach from a node whose api object was deleted", to use an "Expect" instead of a soft error because the test needs to guarantee that the environment is *completely* reset.

**Release note**:

```release-note-none
```

Centralize Capacity discovery of standard resources in Container manager.
Have storage derive node capacity from container manager.
Move certain cAdvisor interfaces to the cAdvisor package in the process.

This patch fixes a bug in container manager where it was writing to a map without synchronization.
Signed-off-by: Vishnu kannan <vishnuk@google.com>

Automatic merge from submit-queue (batch tested with PRs 45610, 47628)

Replace capacity with allocatable to calculate pod resource

It is not accurate to use capacity to do the calculation.



**What this PR does / why we need it**:
The currently cpu resource calculation for a pod in end2end test is incorrect.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes #47627

**Special notes for your reviewer**:
More details about capacity and allocatable:
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node-allocatable.md

**Release note**:

NONE

Automatic merge from submit-queue (batch tested with PRs 45610, 47628)

Add Priority to Kubernetes API

**What this PR does / why we need it**: This is the first in a series of PRs to add priority to Kubernetes API. Subsequent PRs will add priority name resolution to admission controller.


**Release note**:

```release-note
Add PriorityClassName and Priority fields to PodSpec.
```

 * Rename variables and functions to match Go convention.
   For example, UsageError --> UsageErrorf.
 * Remove redundant or unreachable code.
 * Simplify some utility functions (no functionality changes).
 * Fix hanging 'if { return } else { return }' constructs.
 * Fix several incorrect printf verbs.

Remove the built-in kubelet (finally), pick up security fixes.

Automatic merge from submit-queue (batch tested with PRs 47675, 48001)

Encodes ReportPrefix into the generated metrics file names

Ref: https://github.com/kubernetes/kubernetes/issues/44003

Adds the test prefix to be part of the name. Otherwise the same test case running on different images will override each other. Nothing needs to be changed at the node-perf-dash side.

See test run at https://console.cloud.google.com/storage/browser/ygg-gke-dev-bucket/e2e-node-test/ci-kubernetes-node-kubelet-benchmark/10.


**Release note**:
```
None
```

/sig node
/area node-e2e
/assign @Random-Liu

Automatic merge from submit-queue (batch tested with PRs 47675, 48001)

cmd/create_deployment: refactor long function

Refactor the `createDeployment` function under `pkg/kubectl/cmd`.

- [x] Behavior has been extracted to two helper functions.
- [x] Behavior remains identical.
- [x] Logic has been made explicit through function naming and comments.

This is essentially the pattern I've been following in my larger branches (the ones that are pending the merge of #46468). Want to get some design feedback before I get too far away from `master`.

Thanks!

cc @apelisse @mengqiy @droot 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48106, 46761)

removing unnecessary shallow copy on SyncService

**What this PR does / why we need it**: remove shallow copy + avoid using same index name as the nested loop

**Which issue this PR fixes**: fixes #46703 

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 48106, 46761)

s/count/total/ in audit prometheus metrics

Responding to https://github.com/kubernetes/kubernetes/pull/46732#discussion_r124057806, making the metric names conform with the [metric naming convention](https://prometheus.io/docs/practices/naming/#metric-names).

kubernetes/features#22

/cc @ericchiang @ihmccreery

Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Pipe clusterID into gce_loadbalancer_external.go

**What this PR does / why we need it**: Small cleanup for GCE ELB codes.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48002

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Add a failsafe for etcd not returning a connection string

**What this PR does / why we need it**: Removing a kubernetes-master will fail as described on this issue: https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/311

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/311

**Special notes for your reviewer**: This is a two liner defensive code. I am not totally sold on this patch. I might not be the right place to address the above issue. However, solving the problem on the etcd side and updating the interface scope to be unit (as suggested) seems much more involving.

**Release note**:

```
Fix error when removing juju kubernetes-master unit
```

Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Speed up attach/detach controller integration tests

Internal attach/detach controller timers should be configurable and tests should use much shorter values.

`reconcilerSyncDuration` is deliberately left out of `TimerConfig` because it's the only one that's not a constant one, it's configurable by user.

Fixes #47129 

Before:
```
--- PASS: TestPodDeletionWithDswp (63.21s)
--- PASS: TestPodUpdateWithWithADC (13.68s)
--- PASS: TestPodUpdateWithKeepTerminatedPodVolumes (13.55s)
--- PASS: TestPodAddedByDswp (183.01s)
--- PASS: TestPersistentVolumeRecycler (12.55s)
--- PASS: TestPersistentVolumeDeleter (12.54s)
--- PASS: TestPersistentVolumeBindRace (3.51s)
--- PASS: TestPersistentVolumeClaimLabelSelector (12.50s)
--- PASS: TestPersistentVolumeClaimLabelSelectorMatchExpressions (12.54s)
--- PASS: TestPersistentVolumeMultiPVs (3.05s)
--- PASS: TestPersistentVolumeMultiPVsPVCs (4.36s)
--- PASS: TestPersistentVolumeControllerStartup (7.29s)
--- PASS: TestPersistentVolumeProvisionMultiPVCs (5.02s)
--- PASS: TestPersistentVolumeMultiPVsDiffAccessModes (12.48s)
ok  	k8s.io/kubernetes/test/integration/volume	359.727s
```

After:
```
--- PASS: TestPodDeletionWithDswp (3.71s)
--- PASS: TestPodUpdateWithWithADC (3.63s)
--- PASS: TestPodUpdateWithKeepTerminatedPodVolumes (3.70s)
--- PASS: TestPodAddedByDswp (5.68s)
--- PASS: TestPersistentVolumeRecycler (12.54s)
--- PASS: TestPersistentVolumeDeleter (12.55s)
--- PASS: TestPersistentVolumeBindRace (3.55s)
--- PASS: TestPersistentVolumeClaimLabelSelector (12.50s)
--- PASS: TestPersistentVolumeClaimLabelSelectorMatchExpressions (12.52s)
--- PASS: TestPersistentVolumeMultiPVs (3.98s)
--- PASS: TestPersistentVolumeMultiPVsPVCs (3.85s)
--- PASS: TestPersistentVolumeControllerStartup (7.18s)
--- PASS: TestPersistentVolumeProvisionMultiPVCs (5.23s)
--- PASS: TestPersistentVolumeMultiPVsDiffAccessModes (12.48s)
ok  	k8s.io/kubernetes/test/integration/volume	103.267s
```

PV controller tests are the slowest ones now.

@kubernetes/sig-storage-pr-reviews 
/assign @gnufied 

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Reflect kubeadm-specific kubelet changes in the bazel debs

**What this PR does / why we need it**:

Sync this file with the one in kubernetes/release

cc @pipejakob @mikedanese @ixdy 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes: https://github.com/kubernetes/kubeadm/issues/321

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Fix fluentd-gcp configuration to facilitate JSON parsing

There's a bug in https://github.com/kubernetes/kubernetes/pull/45734, because of which each records gets additional field and google-cloud plugin thinks it's not JSON (https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/blob/master/lib/fluent/plugin/out_google_cloud.rb#L569)

Fixes https://github.com/kubernetes/kubernetes/issues/48108

/cc @piosz @fgrzadkowski

Automatic merge from submit-queue

shows how to wire admission control in a sample api server.

**What this PR does / why we need it**:
This PR shows how to wire admission control in a custom api server

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Make big clusters work again after introduction of subnets

This PR does two things: 
  - make IP aliases automatically pick Node IP Range based on number of Nodes,
  - fix logic for starting clusters >4095 Nodes that was broken by introduction of subnets,

cc @wojtek-t @shyamjvs 

```release-note
Setting env var ENABLE_BIG_CLUSTER_SUBNETS=true will allow kube-up.sh to start clusters bigger that 4095 Nodes on GCE.
```

Ref https://github.com/kubernetes/kubernetes/issues/47344

Automatic merge from submit-queue

Insert Cynerva and Kjackal to approvers list

**What this PR does / why we need it**:
Per the membership reviews, we're looking to promote Konstantinos and
George to approvers to help distribute the review/bug load for the `cluster/juju` code
tree.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

**Special notes for your reviewer**:
cc @marcoceppi and @tvansteenburgh 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 47038, 47105)

kuberuntime: cleanup TestGenerateContainerConfig

Followup of #47009, cleanup TestGenerateContainerConfig and remove unused expectedConfig.

/assign @yujuhong

Automatic merge from submit-queue (batch tested with PRs 47038, 47105)

extending DefaultExternalHost for any registred cloud provider

**What this PR does / why we need it**: this PR enables DefaultExternalHost to work with any registered cloud provider.

**Which issue this PR fixes** : fixes #46567 

**Special notes for your reviewer**:

**Release note**:

```release-note
When determining the default external host of the kube apiserver, any configured cloud provider is now consulted
```