Keep using storage v1beta1 until GKE enables it

Automatic merge from submit-queue

Protect kubeproxy deployed via kube-up from system OOMs

This change is necessary until it can be moved to Guaranteed QoS Class.

For #40573

Automatic merge from submit-queue

apiserver: self-signed in-memory cert for loopback

We had complicated logic before to find a matching (SNI- or server-) cert for the loopback device. In the worst case, bootstrapping failed because the provided certs didn't have the binding IP or localhost.

We now pass an artifical `ServerName` "apiserver-loopback-client" with the loopback client and install a self-signed in-memory cert+key into the server for that hostname. The client accepts the corresponding cert.

Fixing https://github.com/kubernetes/kubernetes/issues/41081

Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)

Bump gcr.io/google-containers/rescheduler to v0.2.2

**What this PR does / why we need it**: updates the rescheduler image to one based on busybox instead of ubuntu-slim. Changes for the image were in https://github.com/kubernetes/contrib/pull/2390.

Do you think this merits a release note? I'm leaning towards no.

**Release note**:

```release-note
Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu.
```

cc @timstclair

Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)

add kube-aggregator to hyperkube

Adds kube-aggregator to hyperkube.

@lavalamp @pwittrock @kubernetes/sig-api-machinery-misc @smarterclayton 

Google publishes a hyperkube image somewhere, right?

Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)

Add storage.k8s.io/v1 API

v1 API is direct copy of v1beta1 API. This v1 API gets installed and exposed in this PR, I tested that kubectl can create both v1beta1 and v1 StorageClass.

~~Rest of Kubernetes (controllers, examples,. tests, ...) still use v1beta1 API, I will update it when this PR gets merged as these changes would get lost among generated code.~~ Most parts use v1 API now, it would not compile / run tests without it.

**Release note**:
```
Kubernetes API storage.k8s.io for storage objects is now fully supported and is available as storage.k8s.io/v1. Beta version of the API storage.k8s.io/v1beta1 is still available in this release, however it will be removed in a future Kubernetes release.

Together with the API endpoint, StorageClass annotation "storageclass.beta.kubernetes.io/is-default-class" is deprecated and  "storageclass.kubernetes.io/is-default-class" should be used instead to mark a default storage class. The beta annotation is still working in this release, however it won't be supported in the next one.
```

@kubernetes/sig-storage-misc

Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)

Fix  some typos

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 41854, 41801, 40088, 41590, 41911)

Default storage class for vSphere Fixes #40070

**What this PR does / why we need it**:

Create default storage class for vSphere. This is part of the storage class GA effort https://github.com/kubernetes/features/issues/36

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

fixes #40070

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

Do not run kubelet in test-cmd.sh

The tests are intended to test only `kubectl` commands and do not
require kubelet.

This fixes #41834

/cc @Random-Liu @dchen1107 @nikhiljindal

Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

kubeadm: Add a --ca-cert-path flag to kubeadm join

**What this PR does / why we need it**:

This PR makes it possible to customize where the CA file is written

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
@pires @mikedanese @dmmcquay @jbeda @errordeveloper

Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

Multi master patch

**What this PR does / why we need it**: Corrects a sync files issue present when running in a HA Master configuration. This PR adds logic to syncronize on first deployment for `/etc/kubernetes/serviceaccount.key` which will cause cypto verification failure if not 1:1 on each master unit. Additionally syncs basic_auth and additional files in /srv/kubernetes. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #41019

**Special notes for your reviewer**: This requires PR #41251 as a dependency before merging. 

**Release note**:

```release-note
Juju - K8s master charm now properly keeps distributed master files in sync for an HA control plane.
```

Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

parse single string as group rather than version

**What this PR does / why we need it**:
Prevent bad code generation by adjusting parsing of input.

When client-gen is run with `--input` as an unversioned type, it blows up if the group does not end with a trailing slash.

examples
1. `--input servicecatalog/v1alpha` is group servicecatalog, with version v1alpha1
1. `--input servicecatalog/` is group servicecatalog, with version empty string
1. `--input servicecatalog` is group empty string, with version servicecatalog

I disagree that the second and third cases are different. This adjusts them so they are both case 2.


**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

This change would address the concerns in PR https://github.com/kubernetes/community/pull/343

**Special notes for your reviewer**:
@caesarxuchao @pmorie this does not result in any change in any current code generation, and improves the user experience of client-gen. I ran `hack/update-codegen.sh` and saw no changes in git, nor did anything fail.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

Update i18n tools and process.

@fabianofranz @zen @kubernetes/sig-cli-pr-reviews 

This is an update to the translation process based on feedback from folks.

The main changes are:
   * `msgctx` is being removed from the files.
   * String wrapping and string extraction have been separated.
   * More tools from the `gettext` family of tools are being used
   * Extracted strings are being sorted for canonical ordering
   * A `.pot` template has been added.

Automatic merge from submit-queue

Supports 'ensure exist' class addon in Addon-manager

Fixes #39561, fixes #37047 and fixes #36411. Depends on #40057.

This PR splits cluster addons into two categories:
- Reconcile: Addons that need to be reconciled (`kube-dns` for instance).
- EnsureExists: Addons that need to be exist but changeable (`default-storage-class`).

The behavior for the 'EnsureExists' class addon would be:
- Create it if not exist.
- Users could do any modification they want, addon-manager will not reconcile it.
- If it is deleted, addon-manager will recreate it with the given template.
- It will not be updated/clobbered during upgrade.

As Brian pointed out in [#37048/comment](https://github.com/kubernetes/kubernetes/issues/37048#issuecomment-272510835), this may not be the best solution for addon-manager. Though #39561 needs to be fixed in 1.6 and we might not have enough bandwidth to do a big surgery.

@mikedanese @thockin 

cc @kubernetes/sig-cluster-lifecycle-misc 

---

Tasks for this PR:
- [x] Supports 'ensure exist' class addon and switch to use new labels in addon-manager.
- [x] Updates READMEs regarding the new behavior of addon-manager.
- [x] Updated `test/e2e/addon_update.go` to match the new behavior.
- [x] Go through all current addons and apply the new labels on them regarding what they need.
- [x] Bump addon-manager and update its template files.

Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

Show specific error when a volume is formatted by unexpected filesystem.

kubelet now detects that e.g. xfs volume is being mounted as ext3 because of
wrong volume.Spec.

Mount error is left in the error message to diagnose issues with mounting e.g.
'ext3' volume as 'ext4' - they are different filesystems, however kernel should
mount ext3 as ext4 without errors.

Example kubectl describe pod output:

```
  FirstSeen     LastSeen        Count   From                                    SubobjectPath   Type            Reason          Message
  41s           3s              7       {kubelet ip-172-18-3-82.ec2.internal}                   Warning         FailedMount     MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/aws://us-east-1d/vol-ba79c81d" (spec.Name: "pvc-ce175cbb-6b82-11e6-9fe4-0e885cca73d3") pod "3d19cb64-6b83-11e6-9fe4-0e885cca73d3" (UID: "3d19cb64-6b83-11e6-9fe4-0e885cca73d3") with: failed to mount the volume as "ext4", it's already formatted with "xfs". Mount error: mount failed: exit status 32
Mounting arguments: /dev/xvdba /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1d/vol-ba79c81d ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdba,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
```

Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

Switch scheduler to use generated listers/informers

Where possible, switch the scheduler to use generated listers and
informers. There are still some places where it probably makes more
sense to use one-off reflectors/informers (listing/watching just a
single node, listing/watching scheduled & unscheduled pods using a field
selector).

I think this can wait until master is open for 1.7 pulls, given that we're close to the 1.6 freeze.

After this and #41482 go in, the only code left that references legacylisters will be federation, and 1 bit in a stateful set unit test (which I'll clean up in a follow-up).

@resouer I imagine this will conflict with your equivalence class work, so one of us will be doing some rebasing 😄 

cc @wojtek-t @gmarek  @timothysc @jayunit100 @smarterclayton @deads2k @liggitt @sttts @derekwaynecarr @kubernetes/sig-scheduling-pr-reviews @kubernetes/sig-scalability-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

Disable cgroups-per-qos pending Burstable/cpu.shares being set

Disable cgroups-per-qos to allow kubemark problems to still be resolved.

Re-enable it once the following merge:
https://github.com/kubernetes/kubernetes/pull/41753
https://github.com/kubernetes/kubernetes/pull/41644
https://github.com/kubernetes/kubernetes/pull/41621

Enabling it before cpu.shares is set on qos tiers can cause regressions since Burstable and BestEffort pods are given equal time.

Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

controller: fix requeueing progressing deployments

Drop the secondary queue and add either ratelimited or after the
required amount of time that we need to wait directly in the main
queue. In this way we can always be sure that we will sync back
the Deployment if its progress has yet to resolve into a complete
(NewReplicaSetAvailable) or TimedOut condition.

This should also simplify the deployment controller a bit.

Fixes https://github.com/kubernetes/kubernetes/issues/39785. Once this change soaks, I will move the test out of the flaky suite.

@kubernetes/sig-apps-misc

Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

Split federation-{up,down} from e2e-{up,down}.

The idea is to introduce these as flags to kubetest so we can run these steps independently. I have a separate PR targeting `kubernetes/test-infra/kubetest` to add `--federation_up` and `--federation_down`. So one can run `hack/e2e.go --up --federation_up` to bring up both the clusters and federation control plane. Just `hack/e2e.go --up` to bring up clusters and just `hack/e2e.go --federation_up` to bring up federation control plane.

This PR depends on PR #41601. Please review only the last commit here. Other commits will be reviewed independently in their original PR.

```release-note
NONE
```

Remove extra operations when generating pod sandbox configuration.

Automatic merge from submit-queue

controller: add approvers for rc/rs

Automatic merge from submit-queue

Update owners file to reflect Juju/Charm knowledgable reviewers

**What this PR does / why we need it**:

Several reviewers have expressed confusion on why they're being added to pull requests related to cluster/juju. This is likely because the cluster OWNER wasn't updated from the original when created. This removes those who have expressed confusion to avoid noise in future updates to this directory.

Removed:
 - @pmorie [ref](https://github.com/kubernetes/kubernetes/pull/40814#issuecomment-282107925)
 - @jsafrane [ref](https://github.com/kubernetes/kubernetes/pull/42007#issuecomment-282246574)

Added:
 - @marcoceppi
 - @chuckbutler 
 - @mbruzek  

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Fix zsh completion: unknown file attribute error

**What this PR does / why we need it**:
Fixes zsh completion.

Sourcing the file with `zsh` > 4 resulted in an `unknown file attribute`.
More details at http://stackoverflow.com/questions/37220495/zsh-unknown-file-attribute

sourcing the file with `zsh` > 4 resulted in an `unknown file attribute`.
More details at http://stackoverflow.com/questions/37220495/zsh-unknown-file-attribute

replacing $@ with $* for get_comp_words
as suggested by @sttts worked to resolve the issue

Automatic merge from submit-queue

Add f.close for the opend file

Automatic merge from submit-queue

kubelet config should ignore file start with dots

Fixes: #39156

Ignore files started with dot.