Automatic merge from submit-queue

Support for etcd migration

@xiang90 @timothysc @hongchaodeng

Automatic merge from submit-queue

ClusterAutoscaler-friendly scheduler priority function that promotes well-used nodes

It will help cluster autoscaler to put pods on nodes that are unlikely to be deleted soon due to low usage. Otherwise a pod may be frequently kicked from one node to another. A flag that enables it when CA is on will be added in a separate PR.

Fixes: #28842

Automatic merge from submit-queue

Disable linux/ppc64le compilation by default

Work-around for #30384.

I'm still testing this locally to see if it actually works. The build is slow. (PR Jenkins won't tell us whether this fixes ppc.)

cc @Random-Liu @spxtr @david-mcmahon @luxas

Automatic merge from submit-queue

extract common function in ingress and federation e2e to uil

Separate refactoring existing e2e code part from #29773
1. extract common constants, structs and functions in federated-service.go to federation-util.go, move existing util functions from federated-service.go to federation-util.go
2. extract common constants, structs and functions in ingress.go to ingress-util.go, move existing util functions from ingress.go to ingress-util.go

@quinton-hoole

Automatic merge from submit-queue

Node E2E: Make readiness check handling process exits with 0 exit code.

As is mentioned by @mtaufen:
 "there is a problem with the way service `start` is currently implemented in test/e2e_node/e2e_service.go. If the Kubelet exits with status 0 before the health check completes, cmdErrorChan will be closed and, as a result, nil will be read from that channel, and you will return a nil error from `start`."

This PR changes the logic to:
1) If the err channel returns an error, return the error
2) If the err channel returns a nil, ignore it and continue checking readiness.
3) If the err channel is closed before readiness check succeeds, replace it with `blockCh` and continue checking readiness.

@mtaufen 
/cc @kubernetes/sig-node

Automatic merge from submit-queue

Guarantee that Encode handles nested objects again

pass FederatedServiceName to createServiceOrFail and createService as they are util functions for federated service and federated ingress now.

Automatic merge from submit-queue

Move UTs that block on apiserver to integration tests. 

In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail.  These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.  

This PR simply shuffles those tests to integration tests, which is where they should be. 

/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90

Automatic merge from submit-queue

Increase concurrent GC workers, adjust the polling interval and timeout in e2e test

ref: https://github.com/kubernetes/kubernetes/issues/30759#issuecomment-240853949

Automatic merge from submit-queue

Unset a couple GINKGO_TEST_ARGS in kubekins-e2e

Address #30749

Automatic merge from submit-queue

Check return value when calling ensureDnsRecords

When [lockedUpdateDNSRecords](https://github.com/xiangpengzhao/kubernetes/blob/check_ensureDnsRecords_return_value/federation/pkg/federation-controller/service/servicecontroller.go#L723) calls `ensureDnsRecords`, it should check the return value. If it returns error, the `ensuredCount ` should not increment.

Automatic merge from submit-queue

Revert "Revert "syncNetworkUtil in kubelet and fix loadbalancerSourceRange on GCE

Reverts kubernetes/kubernetes#30729

Fix overlong junit filename prefixes

Automatic merge from submit-queue

Add GUBERNATOR flag which produces g8r link for node e2e tests

When you run 'make tests-e2e-node REMOTE=true GUBERNATOR=true' outputs a URL to view the test results on Gubernator. ~~Should work after my PR for Gubernator is merged.~~

@timstclair

Automatic merge from submit-queue

Bump up glbc version to 0.7.1

stood up should be integration tests, not unit tests.

Automatic merge from submit-queue

more explictly about NoDiskConflicts policy and applicable volume types

partially clarify #29670

@kubernetes/sig-scheduling

Automatic merge from submit-queue

Incorrect branch name for git push command in development.md

the branch name is "my-feature":

    ### Create a branch and make changes

    ```sh
    git checkout -b my-feature

Automatic merge from submit-queue

Quobyte Volume plugin

@quofelix and myself developed a volume plugin for [Quobyte](http://www.quobyte.com) which is a software-defined storage solution. This PR allows Kubernetes users to mount a Quobyte Volume inside their containers over Kubernetes.

Here are some further informations about [Quobyte and Storage for containers](http://www.quobyte.com/containers)

The nested object could be using a different scheme, or want to handle
runtime.Unknown.

Rather than one at a time.

Allows Convert() to reuse the same conversions as ConvertToVersion
without being overly coupled to the version.

Specifically JobTemplate in batch is not in extensions/v1beta1

Introduce an optional interface for callers to encode themselves.

Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.

Automatic merge from submit-queue

Implement dynamic provisioning (beta) of PersistentVolumes via StorageClass

Implemented according to PR #26908. There are several patches in this PR with one huge code regen inside.

* Please review the API changes (the first patch) carefully, sometimes I don't know what the code is doing...

* `PV.Spec.Class` and `PVC.Spec.Class` is not implemented, use annotation `volume.alpha.kubernetes.io/storage-class`

* See e2e test and integration test changes - Kubernetes won't provision a thing without explicit configuration of at least one `StorageClass` instance!

* Multiple provisioning volume plugins can coexist together, e.g. HostPath and AWS EBS. This is important for Gluster and RBD provisioners in #25026

* Contradicting the proposal, `claim.Selector` and `volume.alpha.kubernetes.io/storage-class` annotation are **not** mutually exclusive. They're both used for matching existing PVs. However, only `volume.alpha.kubernetes.io/storage-class` is used for provisioning, configuration of provisioning with `Selector` is left for (near) future.

* Documentation is missing. Can please someone write some while I am out?

For now, AWS volume plugin accepts classes with these parameters:

```
kind: StorageClass
metadata:
  name: slow
provisionerType: kubernetes.io/aws-ebs
provisionerParameters:
  type: io1
  zone: us-east-1d
  iopsPerGB: 10
```

* parameters are case-insensitive
* `type`: `io1`, `gp2`, `sc1`, `st1`. See AWS docs for details
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS volume plugin multiplies this with size of requested volume to compute IOPS of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see AWS docs).
* of course, the plugin will use some defaults when a parameter is omitted in a `StorageClass` instance (`gp2` in the same zone as in 1.3).

GCE:

```
apiVersion: extensions/v1beta1
kind: StorageClass
metadata:
  name: slow
provisionerType: kubernetes.io/gce-pd
provisionerParameters:
  type: pd-standard
  zone: us-central1-a
```

* `type`: `pd-standard` or `pd-ssd`
* `zone`: GCE zone
* of course, the plugin will use some defaults when a parameter is omitted in a `StorageClass` instance (SSD in the same zone as in 1.3 ?).


No OpenStack/Cinder yet

@kubernetes/sig-storage

Automatic merge from submit-queue

kubelet eviction on inode exhaustion

Add support for kubelet to monitor for inode exhaustion of either image or rootfs, and in response, attempt to reclaim node level resources and/or evict pods.

Automatic merge from submit-queue

Fix deadlock possibility in federated informer

On cluster add subinformer locks and tries to add cluster to federated informer. When someone checks if everything is in sync federated informer is locked and then subinformer is inspected what apparently requires a lock. With really bad timing this can create a deadlock.

This PR ensures that there is always at most 1 lock taken in federated informer.

cc: @quinton-hoole @kubernetes/sig-cluster-federation 

Fixes: #30855