I HATE BASH

[Bug Fix] Allow Status Updates for Pods that can be deleted

Automatic merge from submit-queue

Don't wait for the final deletion of pod

The final deletion of the pod depends on kubelet and other components operating correctly. The purpose of this e2e test is verifying the clientset can handle deleteOptions correctly, so waiting for the deletionTimestamp and deletionGraceperiod get set is good enough.

In the long run, we should move this set of e2e tests to integration tests.

Fix #42724 #42646

cc @marun

Automatic merge from submit-queue

add debugging to the client watch test

Adds debugging information for https://github.com/kubernetes/kubernetes/issues/42724.  I suspect that the watch is closing early, but I'd like proof before I consider things like retrying the list and doing another watch to observe the delete.  I'm not even sure that would satisfy the test

It seems like a flaky way to build the test.  Why wouldn't we delete non-gracefully?

@kubernetes/sig-api-machinery-misc @caesarxuchao 
@wojtek-t saw you just hit this if you wanted to take a quick look at the debugging I added.

Automatic merge from submit-queue

Ensure a fixed godep version in hack/*-godep*.sh

No godep pinning asks for trouble when godep changes behaviour once again.

Moreover, call `hack/godep-restore.go` from `hack/update-all-staging.sh`. This was an actual bug.

Automatic merge from submit-queue (batch tested with PRs 42786, 42553)

Updated auto generated protobuf codes.

Generated by `./hack/update-generated-protobuf-dockerized.sh` in Mac.

Automatic merge from submit-queue (batch tested with PRs 42786, 42553)

Updated comments for TaintBasedEvictions.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

Automatic merge from submit-queue (batch tested with PRs 42728, 42278)

[Federation] Create integration test fixture for api

This PR factors a reusable fixture for the federation api server out of the existing integration test.

Targets #40705

cc: @kubernetes/sig-federation-pr-reviews

Automatic merge from submit-queue

Don't try to run hack/verify-staging-* on dirty repository

When the repo is dirty after running all `update-*` scripts in `hack/update-all.sh`, the staging verify scripts still fail. This PR removes these from `hack/update-all.sh`. Instead give useful instructions or continue automatically with `hack/update-all-staging.sh`.

Automatic merge from submit-queue (batch tested with PRs 42762, 42739, 42425, 42778)

Fixed potential OutOfSync of nodeInfo.

The cloned NodeInfo still share the same resource objects in cache; it may make `requestedResource` and Pods OutOfSync, for example, if the pod was deleted, the `requestedResource` is updated by Pods are not in cloned info. Found this when investigating #32531 , but seems not the root cause, as nodeInfo are readonly in predicts & priorities.

Sample codes for `&(*)`:

```
package main

import (
	"fmt"
)

type Resource struct {
	A int
}

type Node struct {
	Res *Resource
}

func main() {
	r1 := &Resource { A:10 }
	n1 := &Node{Res: r1}
	r2 := &(*n1.Res)
	r2.A = 11

	fmt.Printf("%t, %d %d\n", r1==r2, r1, r2)
}
```

Output:

```
true, &{11} &{11}
```

Automatic merge from submit-queue (batch tested with PRs 42762, 42739, 42425, 42778)

kubeadm: update docker version for CE and EE

**What this PR does / why we need it**: Update regex for docker version to also capture new CE and EE versions. 

**Which issue this PR fixes**: fixes #https://github.com/kubernetes/kubeadm/issues/189

**Special notes for your reviewer**: /cc @jbeda @luxas

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 42762, 42739, 42425, 42778)

FakeDockerClient: add creation timestamp

This fixes #42736

Automatic merge from submit-queue (batch tested with PRs 42762, 42739, 42425, 42778)

[Federation][e2e] Use correct default dns name in e2e-testing

After some kubefed changes, the environment variable did not get propagated and we defaulted back to 'federation' instead of 'e2e-federation'. This fixes ongoing service test issues in e2e.

Automatic merge from submit-queue

Add default storageclass tests

**What this PR does / why we need it**:
Adds test cases for using and disabling the default storageclass.

**Release note**:

NONE

Automatic merge from submit-queue (batch tested with PRs 42768, 42760, 42771, 42767)

Stop sandbox container when hit network error.

Fixes https://github.com/kubernetes/kubernetes/issues/42698.

This PR stops the sandbox container when hitting a network error.
This PR also adds a unit test for it.

I'm not sure whether we should try teardown pod network after `SetUpPod` failure. We don't do that in dockertools https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockertools/docker_manager.go#L2276.

@yujuhong @freehan

Automatic merge from submit-queue (batch tested with PRs 42768, 42760, 42771, 42767)

Bump fluentd-gcp to 2.0.2

See https://github.com/kubernetes/contrib/pull/2457

Automatic merge from submit-queue (batch tested with PRs 42768, 42760, 42771, 42767)

Create EnsureExists class addons before Reconcile class addons

From #42757.

The addon-manager creates "Reconcile" class addons before creates "EnsureExists" class addons, which is not the best order. The "EnsureExists" class addons tend to be some default configurations like `default-storage-class` and `default kube-dns ConfigMap` (being added in #42757), and we would like to have these default configurations created before other addons are created.

@mikedanese @bowei 

```release-note
NONE
```

Automatic merge from submit-queue

dockershim: Fix the race condition in ListPodSandbox

In ListPodSandbox(), we
 1. List all sandbox docker containers
 2. List all sandbox checkpoints. If the checkpoint does not have a
    corresponding container in (1), we return partial result based on
    the checkpoint.

The problem is that new PodSandboxes can be created between step (1) and
(2). In those cases, we will see the checkpoints, but not the sandbox
containers. This leads to strange behavior because the partial result
from the checkpoint does not include some critical information. For
example, the creation timestamp'd be zero, and that would cause kubelet's
garbage collector to immediately remove the sandbox.

This change fixes that by getting the list of checkpoints before listing
all the containers (since in RunPodSandbox we create them in the reverse
order).

Automatic merge from submit-queue (batch tested with PRs 42211, 38691, 42737, 42757, 42754)

[Federation] Generate a random nodePort for each service object in e2e tests.

We now run e2e tests in parallel in the CI environment and nodeports are a single available range of numbers for all the service objects, so they have to be unique for each service object.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 42211, 38691, 42737, 42757, 42754)

Adds default kube-dns configmap

From #42746.

Using 1.4 kubelet with 1.6 master is causing upgrade test failures. Because kubelet doesn't understand optional configmaps and there is no kube-dns configmap exist, kubelet will crash.

This PR adds an empty kube-dns configmap as an "EnsureExists" class addon for fixing that.

Note: The addon-manager creates "Reconcile" class addons before creates "EnsureExists" class addons, which is not the order we want. I will soon have another PR to reverse this order.

@bowei @krousey @skriss 

```release-note
none
```

Automatic merge from submit-queue (batch tested with PRs 42211, 38691, 42737, 42757, 42754)

Fix failing kubectl skew tests

Fixes https://github.com/kubernetes/kubernetes/issues/42697

Skew kubectl tests [are broken](https://k8s-testgrid.appspot.com/release-1.6-upgrade-skew#gce-1.6-master-cvm-kubectl-skew&width=80) in "Simple pod should handle in-cluster config" for trying to copy the `kubectl.sh` script instead of the actual `kubectl` binary.


**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 42211, 38691, 42737, 42757, 42754)

Only create the symlink when container log path exists

When using `syslog` logging driver instead of `json-file`, there will not be container log files such as `<containerID-json.log>`. We should not create symlink in this case.

Automatic merge from submit-queue (batch tested with PRs 42211, 38691, 42737, 42757, 42754)

Add more e2e tests for DaemonSet templateGeneration and pod adoption

Depends on #42173

@erictune @kargakis @lukaszo @kubernetes/sig-apps-pr-reviews

This is necessary for kubemark to work correctly.

In ListPodSandbox(), we
 1. List all sandbox docker containers
 2. List all sandbox checkpoints. If the checkpoint does not have a
    corresponding container in (1), we return partial result based on
    the checkpoint.

The problem is that new PodSandboxes can be created between step (1) and
(2). In those cases, we will see the checkpoints, but not the sandbox
containers. This leads to strange behavior because the partial result
from the checkpoint does not include some critical information. For
example, the creation timestamp'd be zero, and that would cause kubelet's
garbage collector to immediately remove the sandbox.

This change fixes that by getting the list of checkpoints before listing
all the containers (since in RunPodSandbox we create them in the reverse
order).

After some kubefed changes, the environment variable did not get propagated
and we defaulted back to 'federation' instead of
'e2e-federation'. This fixes ongoing service test issues in e2e.

Automatic merge from submit-queue

Add more logs to help debugging

This change allows validators to pass warnings as well as errors. This
was needed because of how support for docker 1.13+ and the new EE and CE
versions is currently being handled.