Automatic merge from submit-queue

Kubelet: Refactor all but image related functions in DockerInterface

For #23563.
Based on #23699 and #23844.

Only last 3 commits are new. This PR refactored all functions except image related functions, including:
* CreateExec
* StartExec
* InspectExec
* AttachToContainer
* Logs
* Info
* Version

@kubernetes/sig-node 

Automatic merge from submit-queue

node_e2e: disable serialized image pulls and increase test timeout

This addresses #24605

Automatic merge from submit-queue

disable linkcheck jenkins job

I don't have time to fix linkcheck any soon, so temporarily disable the job.

ref #23162

Automatic merge from submit-queue

Client auth provider plugin framework

Allows client plugins to modify the underlying transport to, for example, add custom authorization headers.

Automatic merge from submit-queue

Rackspace improvements (OpenStack Cinder)

This adds PV support via Cinder on Rackspace clusters. Rackspace Cloud Block Storage is pretty much vanilla OpenStack Cinder, so there is no need for a separate Volume Plugin. Instead I refactored the Cinder/OpenStack interaction a bit (by introducing a CinderProvider Interface and moving the device path detection logic to the OpenStack part).

Right now this is limited to `AttachDisk` and `DetachDisk`. Creation and deletion of Block Storage is not in scope of this PR.

Also the `ExternalID` and `InstanceID` cloud provider methods have been implemented for Rackspace.

Automatic merge from submit-queue

First pass at a GKE large cluster Jenkins job

Runs a 1000 node GKE parallel e2e test. On demand only. We'll add more
tests as I see what actually works - this is going to have some
flakiness on its own.

Automatic merge from submit-queue

Add mpio support for iscsi

This allows the iscsi volume to check if a iscsi device belongs to a mpio device
If it does belong to the device then we make sure we mount the mpio device instead of
the raw device. 
The code is based on the current FibreChannel volume support for mpio

example
/dev/disk/by-path/iqn-example.com.2999 -> /dev/sde
Then we check
/sys/block/[dm-X]/slaves/xx
until we find the [dm-X] containing /dev/sde and mount it

Additional work that can be done in future
1. Add multiple portal support to iscsi
2. Move the FibreChannel volume provider to use the code that has been extracted

Automatic merge from submit-queue

update hack/build-go to build federation/cmd/federated-apiserver as well

federation/cmd/federated-apiserver was added in https://github.com/kubernetes/kubernetes/pull/23509

cc @jianhuiz 

Automatic merge from submit-queue

fix ./cluster/kube-up.sh failed after vagrant halt. (issue #18990)

Runs a 1000 node GKE parallel e2e test. On demand only. We'll add more
tests as I see what actually works - this is going to have some
flakiness on its own.

Automatic merge from submit-queue

call genericapiserver directly instead of going via master in federated-apiserver

I can cleanup more code, but this is the minimum required to unblock https://github.com/kubernetes/kubernetes/pull/23847

cc @jianhuiz 

Automatic merge from submit-queue

Allow components to use protobufs while talking to apiserver.

Automatic merge from submit-queue

remove log spam from nodecontroller

@thockin @quinton-hoole ptal.

Automatic merge from submit-queue

docker daemon complains SHM size must be greater than 0

Fixes https://github.com/kubernetes/kubernetes/issues/24588

I am hitting this on Fedora 23 w/ docker 1.9.1 using systemd cgroup-driver.

```
$ docker version
Client:
 Version:         1.9.1
 API version:     1.21
 Package version: docker-1.9.1-9.gitee06d03.fc23.x86_64
 Go version:      go1.5.3
 Git commit:      ee06d03/1.9.1
 Built:           
 OS/Arch:         linux/amd64

Server:
 Version:         1.9.1
 API version:     1.21
 Package version: docker-1.9.1-9.gitee06d03.fc23.x86_64
 Go version:      go1.5.3
 Git commit:      ee06d03/1.9.1
 Built:           
 OS/Arch:         linux/amd64
```

Not sure why I am on the only one hitting it right now, but putting this out here for comment.

/cc @kubernetes/sig-node @kubernetes/rh-cluster-infra @smarterclayton 

Automatic merge from submit-queue

 Disable heapster job which has been broken for a month

https://github.com/kubernetes/kubernetes/issues/23538

This job is no longer producing a useful signal. http://kubekins.dls.corp.google.com/ shows that the last pass was nearly two months ago. I would like to disable the job until someone has the chance to fix it so we are not wasting jenkins resources, contributing to system instability.

Automatic merge from submit-queue

Make fake client actions use fully qualified resource

The output of a versioned clientset is version object. The fake client used to assume only internal objects will be returned. This PR removes this assumption by making fake actions initialized with a fully qualified resource instead of a resource string.

We have to regenerate fake clients in release_1_2 clientset to let it compile. For the test fakes, we are breaking the backwards compatibility promise.

Part of #24155.

Automatic merge from submit-queue

updates to vagrant.md

Addresses issue #24259; merges in edits from the now deleted version of vagrant.md from the kubernetes.github.io/docs/getting-started-guides directory see PR 
https://github.com/kubernetes/kubernetes.github.io/pull/294Signed-off-by: mikebrow <brownwm@us.ibm.com>

Fix verification script for proto-generation

Automatic merge from submit-queue

make storage enablement, serialization, and location orthogonal

This allows a caller (command-line, config, code) to specify multiple separate pieces of config information regarding storage and have them properly composed at runtime.  The information provided is exposed through interfaces to allow alternate implementations, which allows us to change the expression of the config moving forward.  I also fixed up the types to be correct as I moved through.

The same options still exist, but they're composed slightly differently
 1. specify target etcd servers per Group or per GroupResource
 1. specify storage GroupVersions per Groups or per GroupResource
 1. specify etcd prefixes per GroupVersion or per GroupResource
 1. specify that multiple GroupResources share the same location in etcd
 1. enable GroupResources by GroupVersion or by GroupResource whitelist or GroupResource blacklist

The `storage.Interface` is built per GroupResource by:
 1. find the set of possible storage GroupResource based on the priority list of cohabitators
 1. choose a GroupResource from the set by looking at which Groups have the resource enabled
 1. find the target etcd server, etcd prefix, and storage encoding based on the GroupResource

The API server can have its resources separately enabled, but for now I've kept them linked.

@liggitt I think we need this (or something like it) to be able to go from config to these interfaces.  Given another round of refactoring, we may be able to reshape these to be more forward driving.

@smarterclayton this is important for rebasing and for a seamless 1.2 to 1.3 migration for us.

Automatic merge from submit-queue

Increase provisioning test timeouts.

We've encountered flakes in our e2e infrastructure when kubelet took more than one minute to detach a volume used by a deleted pod.

Let's increase the wait period from 1 to 3 minutes. This slows down the test by 2 minutes, but it makes the test more stable.

In addition, when kubelet cannot detach a volume for 3 minutes, let the test wait for additional recycle controller retry interval (10 minutes) and hope the volume is deleted by then. This should not increase usual test time, it makes the test stable when kubelet is _extremely_ slow when releasing the volume.

Fixes: #24161

Automatic merge from submit-queue

etcd3 store: provide compactor util

What's this PR?
- Provides a util to compact keys in etcd.

Reason:
We want to save the most recent 10 minutes event history. It should be more than enough for slow watchers. It is not number based, so it can tolerate event bursts too. We do not want to save longer since the current storage API cannot take advantage of the multi-version key yet. We might keep a longer history in the future.

Automatic merge from submit-queue

Allow Proxy to be initialized with store

Automatic merge from submit-queue

Remove requirement that Endpoints IPs be IPv4
Signed-off-by: André Martins <aanm90@gmail.com>

Release Note: The `Endpoints` API object now allows IPv6 addresses to be stored.  Other components of the system are not ready for IPv6 yet, and many cloud providers are not IPv6 compatible, but installations that use their own controller logic can now store v6 endpoints.

Generate protobuf marshallers for new apps group