Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)

[Federation] Generate the secret name in kubefed join.

Addresses part of #42324. A follow-up PR will address annotating Federation resources.

```release-note
Remove the `--secret-name` flag from `kubefed join`, instead generating the secret name arbitrarily.
```

Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)

Added InterPodAffinity unit test case with Namespace.

**What this PR does / why we need it**:
Added InterPodAffinity unit test case with Namespace: unit test case for #45098

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A

**Release note**:

```release-note-none
```

Automatic merge from submit-queue (batch tested with PRs 45100, 45152, 42513, 44796, 45222)

node-controller: deflake TestUpdateNodeWithMultiplePods

Automatic merge from submit-queue

Removed duplicated 'default' in kubelet's help message

```release-note
None
```

Revert test_owners.csv after bad rebase

Use absolute time and increase grace duration to 500ms for tests under load.

Automatic merge from submit-queue

Remove GetClientsetForCluster()

The newClusterClientset() has insteaded of GetClientsetForCluster(),
and GetClientsetForCluster() run wrong. Let's remove it.

Automatic merge from submit-queue

Use munged semantic version for side-loaded docker tag

**What this PR does / why we need it**: rather than using the md5sum of the dockerized binary for each side-loaded docker image, use the semantic version (with `+`s replaced with `_`s) for the side-loaded docker images.

The use of the md5sum for the docker tag dates to #6326 2 years ago. I'm not sure why that was chosen, short of it being fairly unique.

My main motivation for changing this is that it makes building the docker images using Bazel's docker rules easier, since the semantic version doesn't depend on the build output.

An added benefit is that the list of images on a running kubernetes cluster is also more straightfoward; rather than a list of opaque, meaningless hexadecimal strings, you get something that indicates the provenance of the image. It'd also be clearer that all of the images came from the same build.

I was able to start a cluster with this change on GCE using both `make quick-release` and `make bazel-release`.

Note that this change has no effect on the tag that's pushed to gcr.io during releases; that's still controlled via `KUBE_IMAGE_DOCKER_TAG`, though we may want to merge this functionality at some point.

@kubernetes/sig-node-pr-reviews is there any reason to stick with using the md5sum strategy? @dchen1107 do you remember why we went with md5sums originally?
cc @spxtr @mikedanese 

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Allow specifying cluster signing ca/key

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Move federated ingress e2e tests that don't pass to serial suite to avoid running it in presubmits.

This is due to a bug described here: https://github.com/kubernetes/ingress/issues/645. 

Federated Ingress e2es fail when there is a high load which is pretty common in the presubmit environment when the PR activity is high. So we are disabling these tests to unblock federation presubmits.

**Release note**:
```release-note
NONE
```

/assign @csbell 

cc @kubernetes/sig-federation-pr-reviews @nicksardo

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Refactor kube-proxy configuration

This is a proof of concept refactoring of the configuration and startup of kube-proxy. Most flags have been removed and replaced by a single config file, specified by `--config`. This is in regards to the component configuration improvement suggestions listed in #32215.

Also during this effort, I discovered that Hyperkube is roughly reimplementing portions of cobra, and that the current cobra command definitions are solely used to generated docs and man pages. I would like to move the individual commands as well as Hyperkube to using cobra, but that is a separate issue and discussion.

cc @mikedanese @liggitt @deads2k @eparis @sttts @smarterclayton @dgoodwin @timothysc

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Move k8s.io/metrics to staging/

This is to break the cyclic dependency in our code base: kubernetes depends on k8s.io/metrics, which depends on kubernetes/staging/client-go.

@DirectXMan12 i actually moved it to staging because we will need the flexibility to update metrics code directly to do many planned refactors, so the copy of metrics in kubernetes has to be the source of truth.

client-gen is not enabled for the code yet, we can enable it after you port your changes to client-gen.

`make generated_files` is enabled for metrics.

Automatic merge from submit-queue

examples/guestbook: remove not useful/inaccurate comments

The comments removed in this patch are either inaccurate or
does not serve any purpose as they're already applies as uncommented
in the tutorial and explained in the walkthrough text.

```release-note
NONE
```
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>

Automatic merge from submit-queue

Add e2e network policy tests.

**What this PR does / why we need it**:
Add set of e2e tests for Network Policy. This has succesfully run againast a deployment using Calico as the network policy provider.

Specifically, adds a new e2e test file (/test/e2e/network_policy.go) which tests TCP connectivity between pods with isolation.

See #25638 for PR that added NetworkPolicy resource.

This PR is a replacement for #27447 .

Automatic merge from submit-queue

Restructure unit tests for more cert/keys.

Just changing the unit tests so there is multiple cert/key pairs to be used.
No functional change, no new tests. Follow on PRs will make more use
of the multiple cert/key pairs.

Makes it useful in testing some scenarios with automatically
signing certificates

Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Node e2e: Remove CRI/non-CRI configs

This depends on https://github.com/kubernetes/test-infra/pull/2363

Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Increase pv test ns deletion timeout to the framework's default.

**What this PR does / why we need it**:
Namespace deletion occasionally exceeds the test's timeout of 3 minutes.  The test framework uses a default timeout of 5 minutes, so just making it match here.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #44790

**Special notes for your reviewer**:

**Release note**:

NONE

Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Increase Dashboard memory limits

**What this PR does / why we need it**: Increases memory requests and limits for Dashboard.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/dashboard/issues/1431

**Special notes for your reviewer**: Dashboard crashes on large clusters, this change should fix that problem.

**Release note**:

```release-note
Increase Dashboard's memory requests and limits
```

Automatic merge from submit-queue

Remove DockerManager from kubelet

This commit deletes code in dockertools that is only used by
DockerManager. A follow-up change will rename and clean up the rest of
the files in this package.

The commit also sets EnableCRI to true if the container runtime is not
rkt. A follow-up change will remove the flag/field and all references to
it.

Add support for configuring kube-proxy via a config file instead of
command line flags.

Automatic merge from submit-queue

adds Ubuntu node e2e test

**What this PR does / why we need it**:

Add node e2e test for Ubuntu image.

**Special notes for your reviewer**:

**Release note**:

```NONE```

Automatic merge from submit-queue (batch tested with PRs 44062, 45165)

Run SSH e2e test only if SSH keys are present

**What this PR does / why we need it**:

Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Support arbitrary alphanumeric strings as prerelease identifiers

**What this PR does / why we need it**: this is basically an extension of #43642, but supports more general prerelease identifiers, per the spec at http://semver.org/#spec-item-9.

These regular expressions are still a bit more restrictive than the SemVer spec allows (we disallow hyphens, and we require the format `-foo.N` instead of arbitrary `-foo.X.bar.Y.bazZ`), but this should support our needs without changing too much more logic or breaking other assumptions.

**Release note**:

```release-note
NONE
```

This commit deletes code in dockertools that is only used by
DockerManager. A follow-up change will rename and clean up the rest of
the files in this package.

The commit also sets EnableCRI to true if the container runtime is not
rkt. A follow-up change will remove the flag/field and all references to
it.

Automatic merge from submit-queue (batch tested with PRs 40774, 41742)

CRI: using typed filed for Apparmor

**What this PR does / why we need it**:

CRI supports AppArmor via annotations [today](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/api/v1alpha1/runtime/api.proto#L237), we should move them to typed and structured fields instead. (refer [here](https://github.com/kubernetes/kubernetes/issues/30819#issuecomment-253369441))

**Which issue this PR fixes** 

Part of #39130.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

don't use build tags to mark integration tests

This is a bad pattern that leads to checked in code that isn't check for compilation. We should avoid this if it doesn't provide value, which it seems like it doesn't.

Automatic merge from submit-queue

prevent panic on setting nil deletion timestamp

Setting a nil deletionstamp on unstructured causes panics.  This is done when preparing for create.

Automatic merge from submit-queue

Fix nil pointer issue when making mounts for container

When rebooting one of the nodes in my colleague's cluster, two panics were discovered:

```
E1216 04:07:00.193058    2394 runtime.go:52] Recovered from panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:58
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:51
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:41
/usr/local/go/src/runtime/asm_amd64.s:472
/usr/local/go/src/runtime/panic.go:443
/usr/local/go/src/runtime/panic.go:62
/usr/local/go/src/runtime/sigpanic_unix.go:24
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:1313
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:1473
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/dockertools/docker_manager.go:1495
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/dockertools/docker_manager.go:2125
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/dockertools/docker_manager.go:2093
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:1971
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:530
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/pod_workers.go:171
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/pod_workers.go:154
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/pod_workers.go:215
/usr/local/go/src/runtime/asm_amd64.s:1998
E1216 04:07:00.275030    2394 runtime.go:52] Recovered from panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:58
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:51
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:41
/usr/local/go/src/runtime/asm_amd64.s:472
/usr/local/go/src/runtime/panic.go:443
/usr/local/go/src/runtime/panic.go:62
/usr/local/go/src/runtime/sigpanic_unix.go:24
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/stats/volume_stat_caculator.go:98
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/stats/volume_stat_caculator.go:63
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:86
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:87
/usr/local/go/src/runtime/asm_amd64.s:1998
```

kubectl version
```
Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.8", GitCommit:"693ef591120267007be359f97191a6253e0e4fb5", GitTreeState:"clean", BuildDate:"2016-09-28T03:03:21Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.8", GitCommit:"693ef591120267007be359f97191a6253e0e4fb5", GitTreeState:"clean", BuildDate:"2016-09-28T02:52:25Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
```

The second panic had already been fixed by #33616 and #34251. Not sure what caused the first nil pointer issue and whether it has been fixed yet in the master branch. Just fix it by ignoring the nil pointer when making mounts.

cc @jingxu97 @yujuhong

Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.

Automatic merge from submit-queue

Add google.com to e2e test only under gce/gke

We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 45110, 45148)

write HostAliases to hosts file

**What this PR does / why we need it**: using the PodSpec's `HostAliases`, we write entries into the Kubernetes-managed hosts file.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43632

**Special notes for your reviewer**:
Previous PRs in this series:
- https://github.com/kubernetes/kubernetes/pull/44572 isolates the logic of creating the file and writing the file
- https://github.com/kubernetes/kubernetes/pull/44641 introduces the `HostAliases` field in PodSpec along with validations

**Release note**:
```release-note
PodSpec's `HostAliases` now write entries into the Kubernetes-managed hosts file.
```

@thockin @yujuhong 

Thanks for reviewing!

Automatic merge from submit-queue (batch tested with PRs 45110, 45148)

Make timeouts in the Kubelet slightly offset to aid debugging

Several of these loops overlap, and when they are the reason a failure
is happening it is difficult to sort them out. Slighly misalign these
loops to make their impact obvious.

We are seeing exactly 2 minute pod worker timeouts in a wide range of test flake scenarios, and I want to be confident we know exactly which one is the culprit.