Automatic merge from submit-queue (batch tested with PRs 50919, 51410, 50099, 51300, 50296)

Remove failure check from deployment controller

@kubernetes/sig-apps-pr-reviews this check is useless w/o automatic rollback so I am removing it.

Automatic merge from submit-queue (batch tested with PRs 50919, 51410, 50099, 51300, 50296)

GCE: Read networkProjectID param

Fixes #48515 

/assign bowei

The first commit is the original PR cherrypicked. The master's kubelet isn't provided a cloud config path, so the project is retrieved via instance metadata. In the GKE case, this project cannot be retrieved by the master and caused an error.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 50919, 51410, 50099, 51300, 50296)

Take mount options to GA by adding PV.spec.mountOptions

**What this PR does / why we need it**: Implements https://github.com/kubernetes/community/pull/771

issue: https://github.com/kubernetes/features/issues/168

**Special notes for your reviewer**:

TODO:
- ~StorageClass mountOptions~

As described in proposal, this adds PV.spec.mountOptions + mountOptions parameter to every plugin that is both provisionable & supports mount options.

(personally, even having done all the work already, i don't agree w/ the proposal that mountOptions should be SC parameter but... :))

**Release note**:

```release-note
Add mount options field to PersistentVolume spec
```

Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Feature gate initializers field

The metadata.initializers field should be feature gated and disabled by default while in alpha, especially since enforcement of initializer permission that keeps users from submitting objects with their own initializers specified is done via an admission plugin most clusters do not enable yet.

Not gating the field and tests caused tests added in https://github.com/kubernetes/kubernetes/issues/51429 to fail on clusters that don't enable the admission plugin.

This PR:
* adds an `Initializers` feature gate, auto-enables the feature gate if the admission plugin is enabled
* clears the `metadata.initializers` field of objects on create/update if the feature gate is not set
* marks the e2e tests as feature-dependent (will follow up with PR to test-infra to enable the feature and opt in for GCE e2e tests)

```release-note
Use of the alpha initializers feature now requires enabling the `Initializers` feature gate. This feature gate is auto-enabled if the `Initialzers` admission plugin is enabled.
```

Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Fix inconsistent Prometheus cAdvisor metrics

**What this PR does / why we need it**:

We need this because otherwise kubelet is exposing different sets of Prometheus metrics that randomly include or do not include containers.

See also https://github.com/google/cadvisor/issues/1704; quoting here:

Prometheus requires that all metrics in the same family have the same labels, so we arrange to supply blank strings for missing labels

The function `containerPrometheusLabels()` conditionally adds various metric labels from container labels - pod name, image, etc. However, when it receives the metrics, Prometheus [checks](https://github.com/prometheus/client_golang/blob/master/prometheus/registry.go#L665) that all metrics in the same family have the same label set, and [rejects](https://github.com/prometheus/client_golang/blob/master/prometheus/registry.go#L497) those that do not.

Since containers are collected in (somewhat) random order, depending on which kind is seen first you get one set of metrics or the other.

Changing the container labels function to always add the same set of labels, adding `""` when it doesn't have a real value, eliminates the issue in my testing.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes #50151

**Special notes for your reviewer**:

I have made the same fix in two places.  I am 98% sure the one in `cadvisor_linux.go` isn't used and indeed cannot be used, but have not gone fully down that rabbit-hole.

**Release note**:
```release-note
Fix inconsistent Prometheus cAdvisor metrics
```

Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Add local storage to downwards API

**Release note**:
```release-note
Add local ephemeral storage to downward API 
```


/assign @NickrenREN

Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Allow zsh completion to be autoloaded by compinit

**What this PR does / why we need it**:
Allows the kubectl zsh autocompletion to be auto loaded by compinit. Had to move the the boilerplate down into the specific shell functions as the compdef needs to be the first line in the definition file.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50560

**Special notes for your reviewer**:

**Release note**:

```release-note
kubectl zsh autocompletion will work with compinit
```

Automatic merge from submit-queue (batch tested with PRs 51471, 50561, 50435, 51473, 51436)

Fix `gcloud compute instance-groups managed list` call

**What this PR does / why we need it**: gcloud 168.0.0 makes the `gcloud compute instance-groups managed list --format='value(instanceGroup)'` call return a URL instead of just the name, which is causing `list-instances` to fail. Switching to `--format='value(name)'` seems to restore the old behavior.

x-ref #49673 

**Release note**:

```release-note
NONE
```

/cc @wojtek-t @mwielgus @shyamjvs @jiayingz @mindprince

Automatic merge from submit-queue (batch tested with PRs 50932, 49610, 51312, 51415, 50705)

Allow running kube-proxy as a DaemonSet when using kube-up.sh on GCE

**What this PR does / why we need it**:
From #23225, this PR adds an option for user to run kube-proxy as a DaemonSet instead of static pods using GCE startup scripts. By default, kube-proxy will run as static pods.

This is the first step for moving kube-proxy into a DaemonSet in GCE, remaining tasks will be tracked on #23225.

**Special notes for your reviewer**:
The last commit are purely for testing out kube-proxy as daemonset via CIs.

cc @kubernetes/sig-network-misc @kubernetes/sig-cluster-lifecycle-misc 

**Release note**:

```release-note
When using kube-up.sh on GCE, user could set env `KUBE_PROXY_DAEMONSET=true` to run kube-proxy as a DaemonSet. kube-proxy is run as static pods by default.
```

Automatic merge from submit-queue (batch tested with PRs 50932, 49610, 51312, 51415, 50705)

Add --request-timeout to kube-apiserver to make global request timeout configurable

**What this PR does / why we need it**:

Make the currently hard coded 60 global request timeout in apiserver configurable via a --request-timeout command line flag.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

https://github.com/kubernetes/kubernetes/issues/51355

**Special notes for your reviewer**:

We plan to backport this to 1.6 and 1.7. To ease the backport work, we've kept this change to a minimum. After this PR we will submit an additional PR to rename `--min-request-timeout` to something that more clearly describes it's purpose, such as `--long-running-request-timeout-min`.

**Release note**:

```release-note
Add --request-timeout to kube-apiserver to make global request timeout configurable.
```

Automatic merge from submit-queue (batch tested with PRs 50932, 49610, 51312, 51415, 50705)

Deprecation warnings for auto detecting cloud providers

**What this PR does / why we need it**:
Adds deprecation warnings for auto detecting cloud providers. As part of the initiative for out-of-tree cloud providers, this feature is conflicting since we're shifting the dependency of kubernetes core into cAdvisor. In the future kubelets should be using `--cloud-provider=external` or no cloud provider at all. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50986

**Special notes for your reviewer**:
NOTE: I still have to coordinate with sig-node and kubernetes-dev to get approval for this deprecation, I'm only opening this PR since we're close to code freeze and it's something presentable.  

**Release note**:
```release-note
Deprecate auto detecting cloud providers in kubelet. Auto detecting cloud providers go against the initiative for out-of-tree cloud providers as we'll now depend on cAdvisor integrations with cloud providers instead of the core repo. In the near future, `--cloud-provider` for kubelet will either be an empty string or `external`. 
```

Automatic merge from submit-queue (batch tested with PRs 50932, 49610, 51312, 51415, 50705)

Add local storage support in Quota

Add local storage(scratch, overlay) support in quota

**Release note**:
```release-note
Add local ephemeral storage support to Quota
```

/cc @ddysher  @jingxu97

Automatic merge from submit-queue (batch tested with PRs 50932, 49610, 51312, 51415, 50705)

Implement StatsProvider interface using cadvisor

Ref: https://github.com/kubernetes/kubernetes/issues/46984

- This PR changes the `StatsProvider` interface in `pkg/kubelet/server/stats` so that it can provide container stats from either cadvisor or CRI, and the summary API can consume the stats without knowing how they are provided.
- The `StatsProvider` struct in the newly added package `pkg/kubelet/stats` implements part of the `StatsProvider` interface in `pkg/kubelet/server/stats`.
- In `pkg/kubelet/stats`,
    - `stats_provider.go`: implements the node level stats and provides the entry point for this package.
    - `cadvisor_stats_provider.go`: implements the container level stats using cadvisor.
    - `cri_stats_provider.go`: implements the container level stats using CRI.
    - `helper.go`: utility functions shared by the above three components.
- There should be no user visible behaviors change in this PR.
- A follow up PR will implement the StatsProvider interface using CRI.

**Release note**:
```
None
```

/assign @yujuhong 
/assign @WIZARD-CXY

Automatic merge from submit-queue

Fix rsync issue when maintaining data containers

**What this PR does / why we need it**:
When we stopped syncing the full .git dir into the container and created a dummy git tree, an issue arose if you're maintaining a data dir volume. The git tree created in the container wouldn't be updated if we make more commits and sync them in, and would cause a dirty tree.

This PR changes the rsync flags so that "H" filtered files are hidden from rsync on the source side, and if they exist on the destination side, they will be deleted.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes a bug introduced in #50417

**Special notes for your reviewer**:
/assign @sttts @dims @ixdy

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Add liggitt to client-go approvers

/assign @lavalamp
/assign @deads2k

Prometheus requires that all metrics in the same family have the same
labels, so we arrange to supply blank strings for missing labels

See https://github.com/google/cadvisor/issues/1704

Automatic merge from submit-queue

Set flexvolumeplugin.host so that it's not nil

@TerraTech @MikaelCluseau  @chakri-nelluri @verult

I assume this line was removed inadvertently, without plugin.host set the flexvolume silently fails at Mount/Attach* time. https://github.com/kubernetes/kubernetes/pull/50843

https://github.com/kubernetes/kubernetes/issues/51123

Please review, thanks!

```release-note
NONE
```

Automatic merge from submit-queue

When faild create pod sandbox record event.

I created pods because of the failure to create a sandbox, but there was no clear message telling me what was the failure, so I wanted to record an event when the sandbox was created.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue

Make threshold for glbc mem-usage scale with nodes in density test

Fixes https://github.com/kubernetes/kubernetes/issues/51433

cc @wojtek-t

Automatic merge from submit-queue

Always check if default labels on node need to be updated in kubelet

**What this PR does / why we need it**:
Nodes join again but maybe OS/Arch/Instance-Type has changed in the meantime.
In this case the kubelet needs to check if the default labels are still correct and if not it needs to update them.

```release-note
Kubelet updates default labels if those are deprecated
```

Automatic merge from submit-queue (batch tested with PRs 51441, 51356, 51460)

Autodetect kubemark Cloud Provider

**What this PR does / why we need it**:
This is needed for CloudProviderId to be properly
filled for Hollow Nodes.

**Release note**:
```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51441, 51356, 51460)

Don't update pvc.status.capacity if pvc is already Bound

As discussed here https://github.com/kubernetes/community/pull/657#discussion_r128008128, in order for `pvc.status.Capacity < pv.Spec.Capcity` to be the mechanism for volume filesystem* resize, the pv controller should stop updating pvc.status.Capacity every resync period.

/assign @jsafrane
/sig storage
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 51441, 51356, 51460)

fix the bad position of code comment

**What this PR does / why we need it**:

The position of code comment is wrong and move it to the right position

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
NONE
```

Automatic merge from submit-queue

Add support to modify precomputed predicate metadata upon adding/removal of a pod

**What this PR does / why we need it**: This PR adds capability to change precomputed predicate metadata and let's us add/remove pods to the precomputed metadata efficiently without the need ot recomputing everything upon addition/removal of pods. This PR is needed as a part of adding preemption logic to the scheduler.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
To make the review process a bit easier, there are three commits. The cleanup commit is only moving code and renaming some functions, without logic changes.

**Release note**:

```release-note
NONE
```
ref/ #47604
ref/ #48646

/assign @wojtek-t 

@kubernetes/sig-scheduling-pr-reviews @davidopp

Automatic merge from submit-queue

Add volume operation metrics to operation executor and PV controller

This PR implements the proposal for high level volume metrics https://github.com/kubernetes/community/pull/809

**Special notes for your reviewer**:

~Differences from proposal:~ all resolved

~"verify_volume" is now "verify_volumes_are_attached" + "verify_volumes_are_attached_per_node" + "verify_controller_attached_volume." Which of them do we want?~

~There is no "mount_device" metric because the MountVolume operation combines MountDevice and mount (plugin.Setup). Do we want to extract the mount_device metric or is it okay to keep mountvolume as one? For attachable volumes, MountDevice is the actual mount and Setup is a bindmount + setvolumeownership. For unattachable, mountDevice does not occur and Setup is an actual mount + setvolumeownership.~

~PV controller metrics I did not implement following the proposal at all. I did not change goroutinemap nor scheduleOperation. Because provisionClaimOperation does not return an error, so it's impossible for the caller to know if there is actually a failure worth reporting. So I manually create a new metric inside the function according to some conditions.~

@gnufied 

I have tested the operationexecutor metrics but not provision & delete. Sample: 
![screen shot 2017-08-02 at 15 01 08](https://user-images.githubusercontent.com/13111288/28889980-a7093526-7793-11e7-9aa9-ad7158be76fa.png)


**Release note**:

```release-note
Add error count and time-taken metrics for storage operations such as mount and attach, per-volume-plugin.
```

This is needed for CloudProviderId to be properly
filled for Hollow Nodes.

Automatic merge from submit-queue

Moved node condition filter into a predicates.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50360

**Release note**:

```release-note
A new predicates, named 'CheckNodeCondition', was added to replace node condition filter. 'NetworkUnavailable', 'OutOfDisk' and 'NotReady' maybe reported as a reason when failed to schedule pods.
```

Automatic merge from submit-queue (batch tested with PRs 49861, 50933, 51380, 50688, 51305)

fix v1 validation return error

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51304 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```