Automatic merge from submit-queue

Kubelet: Fix the description of MaxContainers kubelet flag.

Found this during code review.

The default number has been changed to `-1` and `1`. https://github.com/kubernetes/kubernetes/blob/82c488bd6ed00b7417ee32bf468d70f4dd18c25a/pkg/apis/componentconfig/v1alpha1/defaults.go#L279-L285
@yujuhong 

/cc @saad-ali This PR fixed incorrect doc.

Automatic merge from submit-queue

Fix default Seccomp profile directory

Looks like some of the refactoring caused us to lose the default
directory. Setting that explicitly here.

Fixes #36350

Automatic merge from submit-queue

Cleanup kubelet eviction manager tests

It cleans up kubelet eviction manager tests

Extracted parts of tests that were similar to each other to functions

Automatic merge from submit-queue

Loosened validation on PVC LimitRanger

This PR loosens validation on PVC LimitRanger so that either Min or Max are required, but not both.

Per @derekwaynecarr  https://github.com/openshift/origin/pull/11396#discussion_r84533061

Automatic merge from submit-queue

Add Windows support to kube-proxy

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:
This is the first stab at supporting kube-proxy (userspace mode) on Windows

**Which issue this PR fixes** : 
fixes #30278

**Special notes for your reviewer**:
The MVP uses `netsh portproxy` to redirect traffic from `ServiceIP:ServicePort` to a `LocalIP:LocalPort`. 
For the next version we are expecting to have guidance from Microsoft Container Networking team.

**Limitations**:
Current implementation does not support DNS queries over UDP as `netsh portproxy` currently only supports TCP. We are working with Microsoft to remediate this.

cc: @brendandburns @dcbw 

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

Automatic merge from submit-queue

Add tests for deepcopy of structs

Until https://github.com/kubernetes/kubernetes/pull/35728 merges, we want to at least fuzz/test that deepcopy isn't shallow-copying problematic fields

Automatic merge from submit-queue

Extend test timeout for LB creation in large clusters

This will most probably be necessary to test 3000-node clusters.

Automatic merge from submit-queue

Support persistent volume usage for kubernetes running on Photon Controller platform

**What this PR does / why we need it:**
Enable the persistent volume usage for kubernetes running on Photon platform.
Photon Controller: https://vmware.github.io/photon-controller/

_Only the first commit include the real code change.
The following commits are for third-party vendor dependency and auto-generated code/docs updating._

Two components are added:
pkg/cloudprovider/providers/photon: support Photon Controller as cloud provider
pkg/volume/photon_pd: support Photon persistent disk as volume source for persistent volume

Usage introduction:
a. Photon Controller is supported as cloud provider.
When choosing to use photon controller as a cloud provider, "--cloud-provider=photon --cloud-config=[path_to_config_file]" is required for kubelet/kube-controller-manager/kube-apiserver. The config file of Photon Controller should follow the following usage:

```
[Global]
target = http://[photon_controller_endpoint_IP]
ignoreCertificate = true
tenant = [tenant_name]
project = [project_name]
overrideIP = true
```

b. Photon persistent disk is supported as volume source/persistent volume source.
yaml usage:

```
volumes:
  - name: photon-storage-1
    photonPersistentDisk:
        pdID: "643ed4e2-3fcc-482b-96d0-12ff6cab2a69"
```
pdID is the persistent disk ID from Photon Controller.

c. Enable Photon Controller as volume provisioner.
yaml usage:

```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: gold_sc
provisioner: kubernetes.io/photon-pd
parameters:
  flavor: persistent-disk-gold
```

The flavor "persistent-disk-gold" needs to be created by Photon platform admin before hand.

Automatic merge from submit-queue

Remove mounter flags from cri test configs

The flags are no longer needed.

Automatic merge from submit-queue

build-image fix for running integration-tests locally

**What this PR does / why we need it**: fix for running integration tests locally.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
current situation if users tries to run 
```console
$ make release
```
or 
```console
$ build-tools/run.sh make test-integration
```
output:
```console
+++ [1104 12:29:13] Stopping any currently running rsyncd container
+++ [1104 12:29:14] Running build command...
+++ [1104 12:29:26] Checking etcd is on PATH
/usr/local/bin/etcd
+++ [1104 12:29:26] Starting etcd instance
etcd --advertise-client-urls http://127.0.0.1:2379 --data-dir /tmp.k8s/tmp.FVPV9pGEWB --listen-client-urls http://127.0.0.1:2379 --debug > "/dev/null" 2>/dev/null
Waiting for etcd to come up.
+++ [1104 12:29:26] On try 1, etcd: : http://127.0.0.1:2379
{"action":"set","node":{"key":"/_test","value":"","modifiedIndex":4,"createdIndex":4}}
+++ [1104 12:29:26] Running integration test cases
Running tests for APIVersion: v1,apps/v1beta1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,batch/v2alpha1,certificates.k8s.io/v1alpha1,extensions/v1beta
1,imagepolicy.k8s.io/v1alpha1,policy/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1
+++ [1104 12:29:30] Running tests without code coverage
ok      k8s.io/kubernetes/test/integration/auth 9.670s
ok      k8s.io/kubernetes/test/integration/client       11.181s
ok      k8s.io/kubernetes/test/integration/configmap    0.780s
setting up a handler for /apis
setting up a handler for /api
Server running on port 9090
Error creating cert: mkdir /var/run/kubernetes: permission deniedW1104 12:31:03.340082    7565 handlers.go:50] Authentication is disabled
[restful] 2016/11/04 12:31:03 log.go:30: [restful/swagger] listing is available at https:///swaggerapi/
[restful] 2016/11/04 12:31:03 log.go:30: [restful/swagger] https:///swaggerui/ is mapped to folder /swagger-ui/
F1104 12:31:03.367346    7565 genericapiserver.go:195] unable to load server certificate: open /var/run/kubernetes/apiserver.crt: no such file or directory
goroutine 90 [running]:
k8s.io/kubernetes/vendor/github.com/golang/glog.stacks(0x2200600, 0xc400000000, 0x9c, 0x198)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:766 +0xa5
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).output(0x21e0340, 0xc400000003, 0xc420160600, 0x2058e22, 0x13, 0xc3, 0x0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:717 +0x337
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).printDepth(0x21e0340, 0xc400000003, 0x1, 0xc420825b50, 0x1, 0x1)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:646 +0x126
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).print(0x21e0340, 0xc400000003, 0xc420825b50, 0x1, 0x1)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:637 +0x5a
k8s.io/kubernetes/vendor/github.com/golang/glog.Fatal(0xc420825b50, 0x1, 0x1)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:1125 +0x53
k8s.io/kubernetes/pkg/genericapiserver.preparedGenericAPIServer.Run(0xc42078eb40, 0xc4200d0a80)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:195 +0x294
k8s.io/kubernetes/examples/apiserver.Run(0xc4204a5000, 0xc4200d0a80, 0x0, 0x0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/examples/apiserver/apiserver.go:108 +0xb1f
k8s.io/kubernetes/test/integration/discoverysummarizer.runAPIServer.func1(0xc4204a5000, 0xc4200d0a80, 0xc4201603c0)
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/integration/discoverysummarizer/discoverysummarizer_test.go:74 +0x39
created by k8s.io/kubernetes/test/integration/discoverysummarizer.runAPIServer
        /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/integration/discoverysummarizer/discoverysummarizer_test.go:77 +0x82
FAIL    k8s.io/kubernetes/test/integration/discoverysummarizer  1.463s
```
and so on. few other tests would be failing with same symptoms, because directory missing and non-writable by regular user. 

**Release note**:
```release-note
NONE
```

Currently if developer tries to run integration tests locally,
it would be failing due to missing /var/run/kubernetes inside
cross container image. As tests are run by non-privileged
user, this directory must be pre-created and made user
writable at the time cross build/test container created.

Automatic merge from submit-queue

Expose addon manager's log by logging to file

Fixes #35823.

Use the same way as  how [`kube-proxy`](https://github.com/kubernetes/kubernetes/blob/master/cluster/saltbase/salt/kube-proxy/kube-proxy.manifest) deals with logging. We would be able to check Addon Manager's logs for Jenkins tests after this.

Would like to see the Jenkins test result to examine.

@mikedanese

Automatic merge from submit-queue

cloudprovider/cloudstack: Fix a bug where we assume IP addresses instead of a hostnames

Because of how our test environment was setup, we didn’t notice that we were assuming the load balancer hosts list to always be IP addresses, while they actually are hostnames.

So without this PR, the load balancer code will not work as expected as it will not be able to find the nodes that need to be load balanced.

Also updated some comments and added a check to prevent trying to release a public IP if we don’t have one.

Automatic merge from submit-queue

add e2e test for kubectl in a Pod

Add a e2e test to make sure kubectl can talk to the api server when it is mounted in a pod.
Fixes: #33138

Automatic merge from submit-queue

CRI: Support string user name.

https://github.com/kubernetes/kubernetes/pull/33239 and https://github.com/kubernetes/kubernetes/pull/34811 combined together broke the cri e2e test. https://k8s-testgrid.appspot.com/google-gce#gci-gce-cri

The reason is that:
1) In dockershim and dockertools, we assume that `Image.Config.User` should be an integer. However, sometimes when user build the image with `USER nobody:nobody` or `USER root:root`, the field will become `nobody:nobody` and `root:root`. This makes dockershim to always return error.
2) The new kube-dns-autoscaler image is using `USER nobody:nobody`. (See https://github.com/kubernetes-incubator/cluster-proportional-autoscaler/blob/master/Dockerfile.in#L21)

This doesn't break the normal e2e test, because in dockertools [we only inspect image uid if `RunAsNonRoot` is set](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockertools/docker_manager.go#L2333-L2338), which is just a coincidence. However, in kuberuntime, [we always inspect image uid first](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/kuberuntime_container.go#L141).

This PR adds literal `root` and `nobody` support. One problem is that `nobody` is not quite the same in different OS distros. Usually it should be `65534`, but some os distro doesn't follow that. For example, Fedora is using `99`. (See https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/Q5GCKZ7Q7PAUQW66EV7IBJGSRJWYXBBH/?sort=date)

Possible solution:
* Option 1: ~~Just use `65534`. This is fine because currently we only need to know whether the user is root or not.~~ Actually, we need to pass the user id to runtime when creating a container.
* Option 2: Return the uid as string in CRI, and let kuberuntime handle the string directly.

This PR is using option 1.

@yujuhong @feiskyer 
/cc @kubernetes/sig-node
/cc @MrHohn

Automatic merge from submit-queue

Make GCI nodes mount non tmpfs, ext* & bind mounts using an external mounter 

This PR downloads the stage1 & gci-mounter ACIs as part of cluster bring up instead of downloading them dynamically from gcr.io, which was the cause for #36206.

I have also optimized the containerized mounter to pre-load the mounter image once to avoid fetch latency while using it.

Original PR which got reverted: https://github.com/kubernetes/kubernetes/pull/35821

```release-note
GCI nodes use an external mounter script to mount NFS & GlusterFS storage volumes
```

@mtaufen Node e2e is not re-enabled in this PR.

cc @jingxu97

Automatic merge from submit-queue

Fixing federation secret controller unit test flakiness

Fixes https://github.com/kubernetes/kubernetes/issues/36422

Adding a wait for the secret to be updated in the store to fix flakiness.
It was failing ~once in 3 to 5 runs before this change. I now have had 30 local runs without a failure.

cc @kubernetes/sig-cluster-federation @mwielgus

Automatic merge from submit-queue

Use the new name for cassandra-statefulset.yaml in e2e tests.

Automatic merge from submit-queue

Add retry to node scheduability marking.

Fixes https://github.com/kubernetes/kubernetes/issues/32391

@justinsb @janetkuo @kubernetes/sig-cli

Merge pull request #36040 from bruceauyeung/add-master-address-desc-into-kubeadm-join-and-some-validations

Automatic merge from submit-queue

add master address into kubeadm join help message and some validations

**What this PR does / why we need it**:

1, add master address into kubeadm join help message. looks like :

>Usage:
>  kubeadm join <master address> [flags]

2, when user provides more than one master address, return an error.

3, since `kubeadm join` not only support ip addresses but also host names or domain names, so i delete the word `ip` from error message `must specify master ip address (see --help)`
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>

Automatic merge from submit-queue

Fix find pattern in generating kubernetes src tarball

#36407 wasn't supposed to merge. This fixes a small issue in it.

Automatic merge from submit-queue

specify custom ca file to verify the keystone server

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

Sometimes the keystone server's certificate is self-signed, mainly used for internal development, testing and etc.

For this kind of ca, we need a way to verify the keystone server.

Otherwise, below error will occur.

> x509: certificate signed by unknown authority

This patch provide a way to pass in a ca file to verify the keystone server when starting `kube-apiserver`.

**Which issue this PR fixes** : fixes #22695, #24984

**Special notes for your reviewer**:

**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
```

Automatic merge from submit-queue

Make kubernetes-src.tar.gz contain source code again

**What this PR does / why we need it**: This PR fixes `kubernetes-src.tar.gz` by reviving the logic for `kube-source.tar.gz` from before #30787.

**Which issue this PR fixes**: fixes #36403

@saad-ali @jbeda @zmerlynn

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Update the gci-mounter sha1 number

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Added a make rule `make upload` to audit and automate release artifact
uploads to GCS.
Signed-off-by: Vishnu kannan <vishnuk@google.com>

This reverts commit 402116ae.

Automatic merge from submit-queue

Node E2E: Extend the default ci node e2e test timeout to 1h.

With more and more test added into node e2e, 45m seems to be not enough now.
I saw sometimes the test takes 40+m:
* https://storage.googleapis.com/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/10942/build-log.txt (44m4.917870119s)
* https://storage.googleapis.com/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/10965/build-log.txt (40m2.37254827s)

And sometimes even timeout:
* https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/10968
* https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/10941

Although it's quite likely that the timeout happened because the limit is too tight, we are not 100% sure.
This PR extends the test timeout of regular ci node e2e test to 1h. Let's see whether the timeout will happen again.

@yujuhong

Automatic merge from submit-queue

Fail kubelet creation if swap enabled and configured with memory eviction thresholds

/cc @vishh @derekwaynecarr

**Release note**:

```
This adds an opt-in Kubelet flag (--fail-swap-on) that causes the Kubelet to fail to start if swap is enabled on the node. This is a temporary opt-in, and the Kubelet will by default fail with swap enabled starting in 1.6.0. The KubeletConfiguration equivalent to the flag is FailSwapOn.
```

Automatic merge from submit-queue

register default functions for federation/apis/core

fix issue #35667 and all related e2e test fail on federation.