e2e-runner: don't clean old binaries before upgrade

Run most GCE, GKE, and Kubemark e2e jobs inside Docker

Automatic merge from submit-queue

Start running [Serial] tests in Trusty CI jobs and the release job of Trusty-dev

Adding these tests as requested in https://github.com/kubernetes/kubernetes/issues/23956#issuecomment-207096136

@spxtr Can you review this?
cc/ @dchen1107 @andyzheng0831 

Automatic merge from submit-queue

disable nfsv4 and use nfsv3

fix #22529 

Automatic merge from submit-queue

e2e: adapt kubelet_perf.go to use the new summary metrics API

This commit switch most functions in kubelet_stats.go to use the new API.
However, the functions that perform one-time resource usage retrieval remain
unchanged to be compatible with reource_usage_gatherer.go. They should be
handled separately.

Also, the new summary API does not provide the RSS memory yet, so all memory
checking tests will *always* pass. We plan to add this metrics in the API and
restore the functionality of the test.

Automatic merge from submit-queue

Remove SecurityContextDeny from vagrant setup

This should not be needed in the vagrant setup.

Automatic merge from submit-queue

Make watch cache treat resourceVersion consistent with uncached watch

Fixes #24004

This makes the watch cache handle resourceVersion consistent with an uncached watch API call, and the documented behavior. Watching from resourceVersion=X delivers watch events *after* version X (X is not included):

> // When specified with a watch call, shows changes that occur after that particular version of a resource.
> // Defaults to changes from the beginning of history.
> ResourceVersion string

Automatic merge from submit-queue

Convert the node conformance tests to use dedicated images.

This PR will de-flake the node e2e.

Add an e2e test suite for multi-zone GKE clusters.

This commit switch most functions in kubelet_stats.go to use the new API.
However, the functions that perform one-time resource usage retrieval remain
unchanged to be compatible with reource_usage_gatherer.go. They should be
handled separately.

Also, the new summary API does not provide the RSS memory yet, so all memory
checking tests will *always* pass. We plan to add this metrics in the API and
restore the functionality of the test.

Automatic merge from submit-queue

Move predicates into library

This PR tries to implement #12744 

Any suggestions/ideas are welcome. @davidopp 

current state: integration test fails if including podCount check in Kubelet.
    
DONE:
    1. refactor all predicates: predicates return fitOrNot(bool) and error(Error) in which the latter is of type PredicateFailureError or InsufficientResourceError
    2. GeneralPredicates() is a predicate function, which includes serveral other predicate functions (PodFitsResource, PodFitsHost, PodFitsHostPort). It is registered as one of the predicates in DefaultAlgorithmProvider, and is also called in canAdmitPod() in Kubelet and should be called by other components (like rescheduler, etc if necessary. See discussion in issue #12744
TODO:
    1. determine which predicates should be included in GeneralPredicates()
    2. separate GeneralPredicates() into: a.) GeneralPredicatesEvictPod() and b.) GeneralPredicatesNotEvictPod()
    3. DaemonSet should use GeneralPredicates()

Automatic merge from submit-queue

Preserve int data when unmarshaling

There are several places we use `json.Unmarshal` into an unstructured map (StrategicMergePatch, UnstructuredJSONScheme, many others).

In this scenario, the json package converts all numbers to float64. This exposes many of the int64 fields in our API types to corruption when the unstructured map is marshalled back to json.

A simple example is a pod with an `"activeDeadlineSeconds": 1000000`. Trying to use `kubectl label`, `annotate`, `patch`, etc results in that int64 being converted to a float64, submitted to the server, and the server rejecting it with an error about "cannot unmarshal number 1e+6 into Go value of type int64"

The json package provides a way to defer conversion of numbers (`json.Decoder#UseNumber`), but does not actually do conversions to int or float. This PR makes use of that feature, and post-processes the unmarshalled map to convert json.Number objects to either int64 or float64 values

Automatic merge from submit-queue

Additional go vet fixes

Mostly:
- pass lock by value
- bad syntax for struct tag value
- example functions not formatted properly

Automatic merge from submit-queue

kubele: add RSS memroy to the summary API

Automatic merge from submit-queue

Fix valid CI version regex in parse_and_validate_release_version().

Ref: #23759
cc @ihmccreery 

Automatic merge from submit-queue

examples/guestbook/php-redis: don't reopen stdio fds

/etc/apache2/apache2.conf was configured to log in this way:
> ErrorLog /proc/self/fd/2
> CustomLog /proc/self/fd/1 combined

This causes apache to reopen the already-opened fds. It works fine when
the file descriptors are pipes or ttys but it fails when they are Unix
sockets because sockets cannot be opened with the open() syscall. The
issue happens when apache is connected to systemd-journald, like in the
rkt container run-time.

This patch uses "cat" to directly write to the stdio fds without
reopening them. apache2.conf now looks like:
> ErrorLog "|$/bin/cat 1>&2"
> CustomLog "|/bin/cat" combined

It works both with Docker and rkt (tested with and without
--interactive).

Symptoms:
> [ 2673.478868] apache2-foreground[4]: (6)No such device or address:
> AH00091: apache2: could not open error log file /proc/self/fd/2.

See also: https://github.com/coreos/rkt/issues/2300

-----

/cc @sjpotter @yifan-gu @jonboulle

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Explicitly pin all <= 1.1 e2e jobs, upgrade, soak, and AWS jobs to
the master node and "legacy" e2e runner.

Remove "dirty workspace" check in e2e-runner.sh

Run kubernetes-e2e-g[ck]e-flaky inside Docker

Automatic merge from submit-queue

kube-scheduler: cleanup duplicate GetAlgorithmProvider()

See [here](https://github.com/kubernetes/kubernetes/blob/ae88f08af09b8e721913544dcc2afdc31f98635e/plugin/pkg/scheduler/factory/factory.go#L177-L180).
The check is duplicate since CreateFromProvider will also do it.

Fix upgrade jobs again

Automatic merge from submit-queue

Add flow control pkg

minor fix ref #15634
Refactor pkg names in back off related files

Automatic merge from submit-queue

Make kube2sky and skydns docker images cross-platform

ARM tracking issue: #17981
Continues on: #19216

Make it possible to create `kube2sky` and `skydns` docker images for ARM and other architectures too
Build in a container, so `golang` isn't a dependency
I've preserved the original default behaviour:
 - `skydns`: It just compiles with go on host
 - `kube2sky`: Build an image

@brendandburns @dchen1107 @ArtfulCoder @thockin @fgrzadkowski

Automatic merge from submit-queue

Minor doc edits to development.md 

Noted a few small clarity issues going through the development guide and while it mentioned local builds, it was missing instructions for building on a local go developer environment.  
Signed-off-by: mikebrow <brownwm@us.ibm.com>

Small improvements in conversion generator

Implement a simpler unversioned.Time serialization

DONE:
1. refactor all predicates: predicates return fitOrNot(bool) and error(Error) in which the latter is of type
	PredicateFailureError or InsufficientResourceError. (For violation of either MaxEBSVolumeCount or
        MaxGCEPDVolumeCount, returns one same error type as ErrMaxVolumeCountExceeded)
2. GeneralPredicates() is a predicate function, which includes serveral other predicate functions (PodFitsResource,
        PodFitsHost, PodFitsHostPort). It is registered as one of the predicates in DefaultAlgorithmProvider, and
        is also called in canAdmitPod() in Kubelet and should be called by other components (like rescheduler, etc)
        if necessary. See discussion in issue #12744
3. remove podNumber check from GeneralPredicates
4. HostName is now verified in Kubelet's canAdminPod(). add TestHostNameConflicts in kubelet_test.go
5. add getNodeAnyWay() method in Kubelet to get node information in standaloneMode

TODO:
1. determine which predicates should be included in GeneralPredicates()
2. separate GeneralPredicates() into:
	a. GeneralPredicatesEvictPod() and
	b. GeneralPredicatesNotEvictPod()
3. DaemonSet should use GeneralPredicates()

Automatic merge from submit-queue

Update hack/test-cmd.sh to use tagged, gcr.io images

Migrate hack/test-cmd.sh and related test data to use tagged, gcr.io versions of the images for #13288 and #20836

Automatic merge from submit-queue

Allowing type object in kubectl swagger validation

Allowing `"type": "object"` in kubectl swagger validation.

We will cherrypick this in 1.2
This allows 1.2 kubectl to work with 1.3 api server which will use `"type":"object"` instead of `"type":"any"`.
Ref https://github.com/kubernetes/kubernetes/issues/4700#issuecomment-194719759

cc @bgrant0607 

Externalize immortal namespaces

include error in the fail msg