Automatic merge from submit-queue

Fix unintended change of Service.spec.ports[].nodePort during kubectl apply

Please refer #23551 for more detail. @bgrant0607 I think this simple fix should be ok to reuse nodePort. @thockin ptal.

Release note: Fix unintended change of `Service.spec.ports[].nodePort` during `kubectl apply`.

Automatic merge from submit-queue

Add RC and container pors to scheduler benchmark

Fix #23263

Ref  #24408
However - scheduler throughput is still ~140 initially, whereas in reality we have 35-40. There are still significant difference we should understand.

@hongchaodeng @xiang90 

Automatic merge from submit-queue

ResourceQuota should be last admission controller

For background, see documentation:

http://kubernetes.io/docs/admin/admission-controllers/#resourcequota

Automatic merge from submit-queue

Cluster Verification Framework

I've spent the last few days looking at the general patterns of verification we have that we tend to reuse in the e2es.  Basically, we need 
 
- label filters
- forEach and WaitFor (where forEach doesn't necessarily waitFor anything).
- timeouts
- multiple phases (reusable definition of state)
- an extensible way to define cluster state that can evolve over time in a data object rather than as a set of parameters that have magic semantics

This PR 
- implements the abstract above functionality declaratively, and w/o hidden semantics.
- addresses the sprawling duplicate methods in #23540, so that we can phase out the wrapper methods and replace them with well defined, extensible semantics for cluster state.
- fixes the recently discovered #23730 issue (where kubectl.go is relying on examples.go, which is obviously wacky) by using the new framework to implement forEachPod in just a couple of lines and migrating the wrapper function into framework.go.

There is some cleanup to do here, but this is seemingly working for a couple of use cases that are important (spark,cassandra,...,kubectl) tests. - i played with a few different ideas and this wound up seeming to be the most natural implementation from a usability standpoint... 

in any case, just thought id push this up as a first iteration, open to feedback.

@kubernetes/sig-testing @timothysc 

Automatic merge from submit-queue

Fix DNS test for larger clusters

On GKE, we scale the number of DNS pods based on the cluster size. For
testing on larger clusters, relax the DNS pod check.

Automatic merge from submit-queue

Added check for noexec mount of /tmp on master

Signed CLA.

Automatic merge from submit-queue

Set ownership to fsGroup in gitrepo

Set ownership to fsGroup in the gitrepo volume plugin

@pmorie 

Automatic merge from submit-queue

genericapiserver: Moving more flags to ServerRunOptions

Moving more apiserver flags to generic api server.

With this change, most of the Config is created in genericapiserver.NewConfig(). The plan is to move everything there.
I didnt touch Storage related params as they are being changed in https://github.com/kubernetes/kubernetes/pull/23208.
And I will handle authz and authn in another PR (need to figure out a few things).

cc @lavalamp @kubernetes/sig-api-machinery 

Automatic merge from submit-queue

Removing federated apiserver's dependency on kubernetes api objects

We can explicitly add kubernetes resources that will be supported in cluster federation, but by default, cluster federation server should not depend on kubernetes resources.

cc @jianhuiz 

Automatic merge from submit-queue

Remove dot in generated deepcopy package name

ref https://github.com/kubernetes/kubernetes/pull/20573#issuecomment-212125061

cc @wojtek-t 

I tested with #20573 to verify hack/codegen.sh worked and the generated code compiled.

Automatic merge from submit-queue

Fix session ended hint for kubectl run

Fixes #23602

Before:
```console
$ kubectl run -i --tty busybox --image=busybox 
Waiting for pod default/busybox-3797442026-mt8zk to be running, status is Pending, pod ready: false
Hit enter for command prompt
/ # 
/ # exit
Session ended, resume using ' busybox-3797442026-mt8zk -c busybox -i -t' command when the pod is running
                             ↑
                      (incomplete command)
```

After:
```console
Session ended, resume using 'kubectl attach busybox-3797442026-mt8zk -c busybox -i -t' command when the pod is running
```

@kubernetes/kubectl 

Automatic merge from submit-queue

Remove host port in influxdb-grafana-controller.yaml

Automatic merge from submit-queue

Don't log private SSH key

Log files may have more inclusive permissions than private SSH keys, and as such we should not log the key, even if it looks invalid. I accidentally leaked my key this way when posting e2e test logs.

Automatic merge from submit-queue

Remove disabled mirror pod test in cmd/integration

The new tests for mirror pods are being added in the node e2e suite.

This partially addresses #24440

Automatic merge from submit-queue

add log line before invoking network plugin

cc: @yujuhong 

Automatic merge from submit-queue

Incremental improvements to kubelet e2e tests

- Add keep-alive to ssh connection
- Don't try to stop services on image-based runs
- Increase jenkins ci timeout to 90 minutes to accomadate unpredictable go build times
- Remove spammy log statement

Automatic merge from submit-queue

Bridge off-cluster traffic into services by masquerading.

Allows service IPs to be accessed from outside of the cluster. This requires that the service range be routed to a node (or several nodes) in the cluster.

Addresses #24224.

@thockin @ArtfulCoder 

Automatic merge from submit-queue

Flush conntrack state for removed/changed UDP Services

fixes: #19029

Automatic merge from submit-queue

Allow setting the Host header in a httpGet probe

The new tests for mirror pods are being added in the node e2e suite.

Automatic merge from submit-queue

Add some more info to the Jenkins README.

This is a bit of a work-in-progress, and I'd appreciate feedback on what to add or remove. I'm not sure that I need to say so much about the GCS format, and I should probably say some more about JJB.

@kubernetes/sig-testing 

Automatic merge from submit-queue

Use tagged gcr.io redis image for pkg/api tests

Migrate pkg/api/validation/testdata redis images to use tagged gcr.io version for https://github.com/kubernetes/kubernetes/issues/13288 and https://github.com/kubernetes/kubernetes/issues/20836

Log files may have more inclusive permissions than private SSH keys, and
as such we should not log the key, even if it looks invalid.

Automatic merge from submit-queue

Removing call to update-swagger-spec.sh from update-generated-swagger-docs.sh

Fixes https://github.com/kubernetes/kubernetes/issues/24233

Right now `update-generated-swagger-docs.sh` calls `update-swagger-spec.sh`, but `verify-generated-swagger-docs.sh` does not verify swagger spec (that is done by `verify-swagger-spec.sh`).
Hence, `verify-swagger-spec` breaks if it is called after `verify-generated-swagger-docs`.

Fixing it by removing the call to `update-swagger-spec.sh` from `update-generated-swagger-docs.sh`.
This will require users to run both `update-swagger-spec` and `update-generated-swagger-docs` when they update api types, but they already need to run many more scripts (`update-api-reference-docs`, `update-codegen`). 
People should mostly be running hack/update-all.sh directly :)

Always pull list-resources.sh from head

Automatic merge from submit-queue

Shorten cluster names in GKE Jenkins on Trusty

We identified an issue that the PD tests in GKE Jenkins on Trusty fail because the PD name is longer than the limit of 63 characters. The PD name embeds the "E2E_NAME" env variable exported in the Jenkins job configuration. This PR shortens that string for all GKE Jenkins on Trusty. As a result, the PD name will meet the limit requirement.

Automatic merge from submit-queue

Remove double assignment of KUBE_ROOT

`KUBE_ROOT` got assigned twice in the codegen script.

Automatic merge from submit-queue

Fix goroutine leak in ssh-tunnel healthcheck.

Tunnel healthchecks were not closing the HTTP response body, leading to many open goroutines.

Automatic merge from submit-queue

Bump kubernetes-test-go timeout.

It looks like the run times got more inconsistent because of load on the VM. Adding another Jenkins slave improved things so we're not constantly timing out, but it still gets a little close to timing out at times.

Average runtime is ~45 mins so I went with a 100 min timeout.

Fixes #24285

Automatic merge from submit-queue

Remove soak and disruptive 1.1 Jenkins jobs.

They're both in the kubernetes-jenkins project, not their own. The disruptive one isn't a critical build, and I don't think the soak should be critical at all, since it's never green for a week anyway and I don't think we ever plan for it to be.