Automatic merge from submit-queue

etcd2: watching from 0 returns all initial states as ADDED events

ref:
https://github.com/kubernetes/kubernetes/pull/36797
https://github.com/kubernetes/kubernetes/issues/36545
https://github.com/kubernetes/kubernetes/pull/36561
https://github.com/kubernetes/kubernetes/issues/13969

Since we have made consensus and fixed the behavior in etcd3, we would also change etcd2 to make this uniform and consistent. The end goal is that we would have it explicit on interface docs.

**release note**:
```
etcd2: watching from 0 returns all initial states as ADDED events
```

Automatic merge from submit-queue

Remove integration test dependencies on etcd client

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does**:

Remove integration test dependencies on etcd client

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 37870, 36643, 37664, 37545)

drop startKubelet's return

Since startKubelet() function will always return nil, we donot need to judge its return err

Automatic merge from submit-queue (batch tested with PRs 37870, 36643, 37664, 37545)

cluster/gci: Fix typo

Automatic merge from submit-queue (batch tested with PRs 37870, 36643, 37664, 37545)

Add option to disable federation ingress controller

**What this PR does / why we need it**:
Added an option to enable/disable federation ingress controller as currently federated ingresses doesn't work in environments other than GCE/GKE. Also ignore reconcile config maps if no federated ingresses exist.

**Which issue this PR fixes** 
fixes #33943

@quinton-hoole

**Release note**:
```release-note
Add `--controllers` flag to federation controller manager for enable/disable federation ingress controller
```

Automatic merge from submit-queue

[Federation] Separate the cleanup phases of service and service shards so that service shards can be cleaned up even after the service is deleted elsewhere.

Fixes Federated Service e2e test.

This separation is necessary because "Federated Service DNS should be
able to discover a federated service" e2e test recently added a case
where it deletes the service from federation but not the shards from
the underlying clusters.

Because of the way cleanup was implemented in the AfterEach block
currently, we did not cleanup any of the underlying shards. However,
separating the two phases of the cleanup needs this separation.

cc @kubernetes/sig-cluster-federation @nikhiljindal

Automatic merge from submit-queue (batch tested with PRs 38149, 38156, 38150)

Node E2E: Remove setup-node option

This PR removes `setup-node` option, because:
* It is misleading. `setup-node` doesn't really setup the node, test framework will only put current user into docker user group when it is specified.
* It is not necessary anymore. Because we always run node e2e test as root now, we don't need to do this anymore.

This is a minor cleanup preparing for my test framework refactoring work. Will send out the refactor PR later.

/cc @kubernetes/sig-node

Automatic merge from submit-queue (batch tested with PRs 38149, 38156, 38150)

Remove girishkalele from most places

@matchstick you might need to help here. I am doing this because the bot is trying to create an issue assigned to @girishkalele but it cannot be created as he is not a member of the org any longer.

Automatic merge from submit-queue

Move some lb tests to the slow suite

These tests have been stable, so move the basic ones to slow.

Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)

fix golint errors on master for 1.6

Needs to be merged before https://github.com/kubernetes/test-infra/pull/1299 is merged

updates https://github.com/kubernetes/kubernetes/issues/37254

Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)

Pass full Node objects to provider LoadBalancer methods

Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)

Fixes flake: wait for dns pods terminating after test completed

From #37194. Based on #36600. Please only look at the second commit.

As mentioned in [comment](https://github.com/kubernetes/kubernetes/issues/37194#issuecomment-262007174), "DNS horizontal autoscaling" test does not wait for the additional pods to be terminated and this may lead to the failure of later tests.

This fix adds a wait loop at the end of the serial test to ensure the cluster recovers to the original state. In the non-serial test it does not wait for the additional pods terminating because it will not affect other tests, given they are able to be run simultaneously. Plus wait for pods terminating will take certain amount of time.

Note this only fixes certain case of #37194. I noticed there are other failures irrelevant to dns autoscaler. LIke [this one](https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce-serial/34/).

@bprashanth @Random-Liu

Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)

Add federation-openapi-spec to update-all targets

As `make verify` runs all `hack/verify-*` scripts (with some exceptions), `update-all.sh` should be in sync. It is missing all the federation targets right now, but `verify-federation-openapi-spec.sh` exists. Hence, we add that.

@nikhiljindal @mbohlool what about the other federation update scripts?

Automatic merge from submit-queue (batch tested with PRs 37328, 38102, 37261, 31321, 38146)

Make thirdparty codec able to decode DeleteOptions

Fix #37278. 

Without this PR, the gvk sent to the delegated codec will be the thirdparty one, which is not recognized by the delegated codec (usually api.Codecs).

Since startKubelet() will always return nil, we donot need to judge its return err

Automatic merge from submit-queue (batch tested with PRs 38076, 38137, 36882, 37634, 37558)

[scheduler] Use V(10) for anything which may be O(N*P) logging

Fixes #37014

This PR makes sure that logging statements which are capable of being called on a perNode / perPod basis (i.e. non essential ones that will just clog up logs at large scale) are at V(10) level.

I dreamt of a levenstein filter that built a weak map of word frequencies and alerted once log throughput increased w/o varying information content....  but then I woke up and realized this is probably all we really need for now :)

Automatic merge from submit-queue (batch tested with PRs 38076, 38137, 36882, 37634, 37558)

Make logging for gcl e2e test more verbose

To help debug https://github.com/kubernetes/kubernetes/issues/37241

CC @piosz

Automatic merge from submit-queue (batch tested with PRs 38076, 38137, 36882, 37634, 37558)

Allow backendpools in Azure Load Balancers which are not owned by cloud provider

**What this PR does / why we need it**: It fixes #36880

**Which issue this PR fixes**: fixes #36880

**Special notes for your reviewer**:

**Release note**:
```release-note
Allow backendpools in Azure Load Balancers which are not owned by cloud provider
```

Instead of bailing out when we find another backend pool, we just ignore
other backend pools and add ours to the list of existing.

Fixes #36880

Automatic merge from submit-queue (batch tested with PRs 38076, 38137, 36882, 37634, 37558)

glusterfs: Fix all gid types to int to prevent failures on 32bit systems

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

The glusterfs dynamic provisioner with GID security has an issue on 32 bit systems.
This fixes that issue by forcing all gid types to int internally.
<!--
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
-->

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
Fix the glusterfs dynamic provisioner for 32bit systems by limiting the gids to type int internally, and allowing 2147483647 as the highest GID.
```

This makes all types int until we hand the GID to heketi/gluster,
at which point it's converted to int64.

It also limits the maximum usable GID ti math.MaxInt32 = 2147483647.
Signed-off-by: Michael Adam <obnox@redhat.com>

Automatic merge from submit-queue (batch tested with PRs 38076, 38137, 36882, 37634, 37558)

Requeue replica sets for availability checks

See https://github.com/kubernetes/kubernetes/issues/35355#issuecomment-264746268 for more details

@kubernetes/deployment @smarterclayton

Disable "overlapping deployment" test on GKE temporarily. #32947

This also updates the maintainers list and reassigns his tests

root.

Signed-off-by: Jess Frazelle <acidburn@google.com>

Automatic merge from submit-queue (batch tested with PRs 38111, 38121)

remove rbac super user

Cleaning up cruft and duplicated capabilities as we transition from RBAC alpha to beta.  In 1.5, we added a secured loopback connection based on the `system:masters` group name.  `system:masters` have full power in the API, so the RBAC super user is superfluous.

The flag will stay in place so that the process can still launch, but it will be disconnected.

@kubernetes/sig-auth

Automatic merge from submit-queue (batch tested with PRs 38111, 38121)

Prevent SIGPIPE errors causing the version check line to fail

The way we use pipe in hack/lib/etcd.sh when processing the version command can lead to
141 failues on some systems. Using a prefix of tail can prevent this.

Fixes #38109

This makes all types int until we hand the GID to heketi/gluster,
at which point it's converted to int64.

It also limits the maximum usable GID ti math.MaxInt32 = 2147483647.
Signed-off-by: Michael Adam <obnox@redhat.com>

Automatic merge from submit-queue (batch tested with PRs 36352, 36538, 37976, 36374)

etcd3: have prefix always prepended

ref: #36290

Previously, the prefix behavior is "sometimes prefixing". If the prefix already exists for the resource path, it will ignore.
With this PR, we make sure that prefix is always prepended in etcd3 storage backend. See the discussion in #36290

Automatic merge from submit-queue (batch tested with PRs 36352, 36538, 37976, 36374)

demonstrate separation of controller intializers

Currently, controllers are all initialized in a monster method that make it difficult to individually pick out whether there are side-effects, difficult to group related controllers for selective enablement, and impossible to determine if there are hidden dependencies.

This pull demonstrates how we can break apart the monolith and start start the process of grouping and naming controllers for selective enablement.  In addition, the use of a map will help expose dependency ordering amongst these controllers and the separate methods will make it a lot harder to have side effects.

This also moves us closer to being able to author reflective unit tests that help ensure that basic RBAC bootstrap roles are at least present, even if they aren't correct.

@nikhiljindal since you were looking at the federation controller manager
@sttts since we're looking at trying out RBAC on these.

Automatic merge from submit-queue (batch tested with PRs 36352, 36538, 37976, 36374)

Check in YAML versions of bootstrap roles/rolebindings

Allows auditing changes to bootstrap roles over time, preventing accidental changes, and gives a place for people to pull bootstrap roles to load directly

Automatic merge from submit-queue (batch tested with PRs 36352, 36538, 37976, 36374)

test: update deployment helper to return better error messages

@kubernetes/deployment the problem with https://github.com/kubernetes/kubernetes/issues/36270 is that the selector key is never added in the deployment but this change would make it clearer.