Closes: #44972

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Allow specifying cluster signing ca/key

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Move federated ingress e2e tests that don't pass to serial suite to avoid running it in presubmits.

This is due to a bug described here: https://github.com/kubernetes/ingress/issues/645. 

Federated Ingress e2es fail when there is a high load which is pretty common in the presubmit environment when the PR activity is high. So we are disabling these tests to unblock federation presubmits.

**Release note**:
```release-note
NONE
```

/assign @csbell 

cc @kubernetes/sig-federation-pr-reviews @nicksardo

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Refactor kube-proxy configuration

This is a proof of concept refactoring of the configuration and startup of kube-proxy. Most flags have been removed and replaced by a single config file, specified by `--config`. This is in regards to the component configuration improvement suggestions listed in #32215.

Also during this effort, I discovered that Hyperkube is roughly reimplementing portions of cobra, and that the current cobra command definitions are solely used to generated docs and man pages. I would like to move the individual commands as well as Hyperkube to using cobra, but that is a separate issue and discussion.

cc @mikedanese @liggitt @deads2k @eparis @sttts @smarterclayton @dgoodwin @timothysc

Automatic merge from submit-queue (batch tested with PRs 45077, 45180, 34727, 45079, 45177)

Move k8s.io/metrics to staging/

This is to break the cyclic dependency in our code base: kubernetes depends on k8s.io/metrics, which depends on kubernetes/staging/client-go.

@DirectXMan12 i actually moved it to staging because we will need the flexibility to update metrics code directly to do many planned refactors, so the copy of metrics in kubernetes has to be the source of truth.

client-gen is not enabled for the code yet, we can enable it after you port your changes to client-gen.

`make generated_files` is enabled for metrics.

Automatic merge from submit-queue

examples/guestbook: remove not useful/inaccurate comments

The comments removed in this patch are either inaccurate or
does not serve any purpose as they're already applies as uncommented
in the tutorial and explained in the walkthrough text.

```release-note
NONE
```
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>

Automatic merge from submit-queue

Add e2e network policy tests.

**What this PR does / why we need it**:
Add set of e2e tests for Network Policy. This has succesfully run againast a deployment using Calico as the network policy provider.

Specifically, adds a new e2e test file (/test/e2e/network_policy.go) which tests TCP connectivity between pods with isolation.

See #25638 for PR that added NetworkPolicy resource.

This PR is a replacement for #27447 .

Automatic merge from submit-queue

Restructure unit tests for more cert/keys.

Just changing the unit tests so there is multiple cert/key pairs to be used.
No functional change, no new tests. Follow on PRs will make more use
of the multiple cert/key pairs.

Makes it useful in testing some scenarios with automatically
signing certificates

Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Node e2e: Remove CRI/non-CRI configs

This depends on https://github.com/kubernetes/test-infra/pull/2363

Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Increase pv test ns deletion timeout to the framework's default.

**What this PR does / why we need it**:
Namespace deletion occasionally exceeds the test's timeout of 3 minutes.  The test framework uses a default timeout of 5 minutes, so just making it match here.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #44790

**Special notes for your reviewer**:

**Release note**:

NONE

Automatic merge from submit-queue (batch tested with PRs 43884, 44712, 45124, 43883)

Increase Dashboard memory limits

**What this PR does / why we need it**: Increases memory requests and limits for Dashboard.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/dashboard/issues/1431

**Special notes for your reviewer**: Dashboard crashes on large clusters, this change should fix that problem.

**Release note**:

```release-note
Increase Dashboard's memory requests and limits
```

Automatic merge from submit-queue

Remove DockerManager from kubelet

This commit deletes code in dockertools that is only used by
DockerManager. A follow-up change will rename and clean up the rest of
the files in this package.

The commit also sets EnableCRI to true if the container runtime is not
rkt. A follow-up change will remove the flag/field and all references to
it.

Add support for configuring kube-proxy via a config file instead of
command line flags.

Automatic merge from submit-queue

adds Ubuntu node e2e test

**What this PR does / why we need it**:

Add node e2e test for Ubuntu image.

**Special notes for your reviewer**:

**Release note**:

```NONE```

Automatic merge from submit-queue (batch tested with PRs 44062, 45165)

Run SSH e2e test only if SSH keys are present

**What this PR does / why we need it**:

Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

Support arbitrary alphanumeric strings as prerelease identifiers

**What this PR does / why we need it**: this is basically an extension of #43642, but supports more general prerelease identifiers, per the spec at http://semver.org/#spec-item-9.

These regular expressions are still a bit more restrictive than the SemVer spec allows (we disallow hyphens, and we require the format `-foo.N` instead of arbitrary `-foo.X.bar.Y.bazZ`), but this should support our needs without changing too much more logic or breaking other assumptions.

**Release note**:

```release-note
NONE
```

This commit deletes code in dockertools that is only used by
DockerManager. A follow-up change will rename and clean up the rest of
the files in this package.

The commit also sets EnableCRI to true if the container runtime is not
rkt. A follow-up change will remove the flag/field and all references to
it.

Automatic merge from submit-queue (batch tested with PRs 40774, 41742)

CRI: using typed filed for Apparmor

**What this PR does / why we need it**:

CRI supports AppArmor via annotations [today](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/api/v1alpha1/runtime/api.proto#L237), we should move them to typed and structured fields instead. (refer [here](https://github.com/kubernetes/kubernetes/issues/30819#issuecomment-253369441))

**Which issue this PR fixes** 

Part of #39130.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

don't use build tags to mark integration tests

This is a bad pattern that leads to checked in code that isn't check for compilation. We should avoid this if it doesn't provide value, which it seems like it doesn't.

Automatic merge from submit-queue

prevent panic on setting nil deletion timestamp

Setting a nil deletionstamp on unstructured causes panics.  This is done when preparing for create.

Automatic merge from submit-queue

Fix nil pointer issue when making mounts for container

When rebooting one of the nodes in my colleague's cluster, two panics were discovered:

```
E1216 04:07:00.193058    2394 runtime.go:52] Recovered from panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:58
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:51
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:41
/usr/local/go/src/runtime/asm_amd64.s:472
/usr/local/go/src/runtime/panic.go:443
/usr/local/go/src/runtime/panic.go:62
/usr/local/go/src/runtime/sigpanic_unix.go:24
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:1313
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:1473
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/dockertools/docker_manager.go:1495
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/dockertools/docker_manager.go:2125
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/dockertools/docker_manager.go:2093
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:1971
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/kubelet.go:530
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/pod_workers.go:171
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/pod_workers.go:154
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/pod_workers.go:215
/usr/local/go/src/runtime/asm_amd64.s:1998
E1216 04:07:00.275030    2394 runtime.go:52] Recovered from panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:58
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:51
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:41
/usr/local/go/src/runtime/asm_amd64.s:472
/usr/local/go/src/runtime/panic.go:443
/usr/local/go/src/runtime/panic.go:62
/usr/local/go/src/runtime/sigpanic_unix.go:24
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/stats/volume_stat_caculator.go:98
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/stats/volume_stat_caculator.go:63
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:86
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/util/wait/wait.go:87
/usr/local/go/src/runtime/asm_amd64.s:1998
```

kubectl version
```
Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.8", GitCommit:"693ef591120267007be359f97191a6253e0e4fb5", GitTreeState:"clean", BuildDate:"2016-09-28T03:03:21Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.8", GitCommit:"693ef591120267007be359f97191a6253e0e4fb5", GitTreeState:"clean", BuildDate:"2016-09-28T02:52:25Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
```

The second panic had already been fixed by #33616 and #34251. Not sure what caused the first nil pointer issue and whether it has been fixed yet in the master branch. Just fix it by ignoring the nil pointer when making mounts.

cc @jingxu97 @yujuhong

Not all CI systems support ssh keys to be present on the node. This
supports the case where "local" provider is being used when running
e2e test, but the environment does not have a SSH key.

Automatic merge from submit-queue

Add google.com to e2e test only under gce/gke

We should limit the lookup/resolve for google.com when
provider is gce or gke. We should be able to run the
test in environments where this is not allowed or not
available.



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 45110, 45148)

write HostAliases to hosts file

**What this PR does / why we need it**: using the PodSpec's `HostAliases`, we write entries into the Kubernetes-managed hosts file.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43632

**Special notes for your reviewer**:
Previous PRs in this series:
- https://github.com/kubernetes/kubernetes/pull/44572 isolates the logic of creating the file and writing the file
- https://github.com/kubernetes/kubernetes/pull/44641 introduces the `HostAliases` field in PodSpec along with validations

**Release note**:
```release-note
PodSpec's `HostAliases` now write entries into the Kubernetes-managed hosts file.
```

@thockin @yujuhong 

Thanks for reviewing!

Automatic merge from submit-queue (batch tested with PRs 45110, 45148)

Make timeouts in the Kubelet slightly offset to aid debugging

Several of these loops overlap, and when they are the reason a failure
is happening it is difficult to sort them out. Slighly misalign these
loops to make their impact obvious.

We are seeing exactly 2 minute pod worker timeouts in a wide range of test flake scenarios, and I want to be confident we know exactly which one is the culprit.

Automatic merge from submit-queue

Create a docker image for the cloud controller manager

**What this PR does / why we need it**:

If you want to test the cloud-controller-manager our currently, you have to grab the binary from the CI or the release builds, put it in a Dockerfile, build the image and possibly upload the image to a registry, only to be able to test the new cloud stuff out. This is overly complex and should just work like any other component like the core controller manager that has its own docker image pushed automatically for it.

This PR makes the release scripts create an image for the cloud controller manager.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Create and push a docker image for the cloud-controller-manager
```

cc @thockin @wlan0 @roberthbailey @mikedanese

Automatic merge from submit-queue

README.md: Update outdated links

**What this PR does / why we need it**:
the PR aims to update some links.
Some links with "#" would not redirect to right point of pages.
Other links without "#" can work, but they are outdated. I change them by the way.

**Special notes for your reviewer**:

**Release note**:

```release-note
```
none

Automatic merge from submit-queue

Align Extender's validation with prioritizers.

**What this PR does / why we need it**:
Align Extender's validation with prioritizers.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A

**Release note**:

```release-note-none
```

Automatic merge from submit-queue

Removed TODO that renaming ReplicationManager.

**What this PR does / why we need it**:
Updated comments to avoid potential PRs on renaming.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # N/A

**Release note**:

```release-note-none
```

Automatic merge from submit-queue

Retry calls we report config changes quickly.

**What this PR does / why we need it**: In Juju deployments of Kubernetes the status of the charms is updated when a status-update is triggered periodically. As a result changes in config variables may take up to 10 minutes to be reflected on the charms status. See bug below.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/263

**Special notes for your reviewer**:

**Release note**:

```
Kubernetes clusters deployed with Juju pick up config changes faster.
```

Automatic merge from submit-queue

Ignore IsNotFound error

IsNotFound error is fine since that means the object is
deleted already, so we should check err and ignore err
before returning.

Automatic merge from submit-queue

Wait for controller events in resource conflict test

Fixes #45137 

same fix as #44996, missed that there was a second test involving resource conflicts