Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)

add nrpe-external-master relation to kubernetes-master and kubernetes-worker

**What this PR does / why we need it**:

This PR adds an an nrpe-external-master relation to the kubernetes-worker, kubernetes-master and kubeapi-load-balancer charms. This is needed to monitor the state of the workers, the masters and the load-balancers via Nagios.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:

https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/165

**Special notes for your reviewer**:

Original work by @axinojolais in PR #40897. All I've done is squash commits on his behalf.

**Release note**:

```release-note
The kubernetes-master, kubernetes-worker and kubeapi-load-balancer charms have gained an nrpe-external-master relation, allowing the integration of their monitoring in an external Nagios server.
```

Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)

kubectl drain: make code reusable

DrainOptions requires a few fields to be set, and the expectation is
that these are set as part of construction of the object.  If they are
set, then the drain code can be reused in other kubernetes projects.

This does not create a contract that DrainOptions should fulfill going
forwards, any more than any of the other types that happen to be exposed
are part of the contract.  Instead, this merely makes use outside the
package possible.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)

Increase Min Timeout for kill pod

Should mitigate #41347, which describes flakes in the inode eviction test due to "GracePeriodExceeded" errors.

When we use gracePeriod == 0, as we do in eviction, the pod worker currently sets a timeout of 2 seconds to kill a pod.
We are hitting this timeout fairly often during eviction tests, causing extra pods to be evicted (since the eviction manager "fails" to evict that pod, and kills the next one).

This PR increases the timeout from 2 seconds to 4, although we could increase it even more if we think that would be appropriate.

cc @yujuhong @vishh @derekwaynecarr

Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)

Release 3.0.17 etcd image

Automatic merge from submit-queue

fix kubectl describe pod, show tolerationSeconds

**What this PR does / why we need it**:
tolerationSeconds is now not shown in kubectl describe resutl, this PR is to fix it.

With this fix, pod toleration with tolerationSeconds would like below:
```yaml
Name:           bar
Namespace:      foo
Node:           /
Labels:         <none>
Status:
IP:
Controllers:    <none>
Containers:     <none>
No volumes.
QoS Class:
Node-Selectors: <none>
Tolerations:    key1=value1
				key2=value2:NoSchedule
				key3=value3:NoExecute for 300s
```


**Which issue this PR fixes** : 
Related issue: #1574
Related PR: #39469

**Special notes for your reviewer**:

**Release note**:

```release-note
make kubectl describe pod show tolerationSeconds
```

Automatic merge from submit-queue (batch tested with PRs 42216, 42136, 42183, 42149, 36828)

Set custom threshold for memory eviction test

I am hoping this helps with memory eviction flakes, e.g. https://github.com/kubernetes/kubernetes/issues/32433 and https://github.com/kubernetes/kubernetes/issues/31676

/cc @derekwaynecarr @calebamiles @dchen1107

Automatic merge from submit-queue (batch tested with PRs 42216, 42136, 42183, 42149, 36828)

Check infra container image existence before pulling.

Fixes https://github.com/kubernetes/kubernetes/issues/42040.

This PR:
* Fixes https://github.com/kubernetes/kubernetes/issues/42040 by checking image existence before pulling.
* Add unit test for it.
* Fix a potential panic at https://github.com/kubernetes/kubernetes/compare/master...Random-Liu:check-infra-container-image-existence?expand=1#diff-e2eefa11d78ba95197ce406772c18c30R421.

@yujuhong

Automatic merge from submit-queue (batch tested with PRs 42216, 42136, 42183, 42149, 36828)

Fix bool parse warning in apimachinery

```golang
var DefaultConverter = NewConverter(parseBool(os.Getenv("KUBE_PATCH_CONVERSION_DETECTOR")))
func parseBool(key string) bool {
	value, err := strconv.ParseBool(key)
	if err != nil {
		utilruntime.HandleError(fmt.Errorf("Couldn't parse '%s' as bool for unstructured mismatch detection", key))
	}
	return value
}
````

leading to

```
W0227 10:06:01.037] E0227 10:06:01.023502   16550 converter.go:87] Couldn't parse '' as bool for unstructured mismatch detection
```

Automatic merge from submit-queue (batch tested with PRs 42216, 42136, 42183, 42149, 36828)

[Federation] Remove federat{ed,ion} prefixes from e2e file names since they are all now scoped under the e2e_federation package.

This is purely cosmetic.

```release-note
NONE
```

cc @kubernetes/sig-federation-pr-reviews

Automatic merge from submit-queue

Don't filter items when resources requested by name

Add tracking on resource.Builder if a "named" item is requested (from
file, stream, url, or resource args) and use that in `get` to accurately
determine whether to filter resources. Add tests.

Fixes #41150, #40492

```release-note
Completed pods should not be hidden when requested by name via `kubectl get`.
```

Automatic merge from submit-queue (batch tested with PRs 42200, 39535, 41708, 41487, 41335)

[Federation][Kubefed] Flag cleanup

This PR is for the issue https://github.com/kubernetes/kubernetes/issues/41333

**Special notes for your reviewer**:
@marun @madhusudancs 


**Release note**:

```
Kubefed init unlearned the following flags:
--storage-backend

Users should instead use the following flag to pass additional arguments:
--apiserver-arg-overrides to api server
```

Automatic merge from submit-queue (batch tested with PRs 42200, 39535, 41708, 41487, 41335)

Update kube-proxy support for Windows

**What this PR does / why we need it**:

The kube-proxy is built upon the sophisticated iptables NAT rules.  Windows does not have an equivalent capability.  This introduces a change to the architecture of the user space mode of the Windows version of kube-proxy to match the capabilities of Windows.

The proxy is organized around service ports and portals.  For each service a service port is created and then a portal, or iptables NAT rule, is opened for each service ip, external ip, node port, and ingress ip.  This PR merges the service port and portal into a single concept of a "ServicePortPortal" where there is one connection opened for each of service IP, external ip, node port, and ingress IP.

This PR only affects the Windows kube-proxy.  It is important for the Windows kube-proxy because it removes the limited portproxy rule and RRAS service and enables full tcp/udp capability to services.

**Special notes for your reviewer**:

**Release note**:

```
Add tcp/udp userspace proxy support for Windows.
```

Automatic merge from submit-queue (batch tested with PRs 42200, 39535, 41708, 41487, 41335)

Add support for statefulset spreading to the scheduler

**What this PR does / why we need it**:
The scheduler SelectorSpread priority funtion didn't have the code to spread pods of StatefulSets. This PR adds StatefulSets to the list of controllers that SelectorSpread supports.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #41513

**Special notes for your reviewer**:

**Release note**:

```release-note
Add the support to the scheduler for spreading pods of StatefulSets.
```

Automatic merge from submit-queue

Portworx Volume Plugin

**What this PR does / why we need it**:
This PR adds a new volume plugin in Kubernetes for Portworx Volumes. Portworx pools available block storage on nodes and carves out volumes from it. More info about Portworx can be found [here](http://docs.portworx.com) The volume plugin supports
- PVs
- PVCs
- Dynamic Provisioning using Storage Classes

**Special notes for your reviewer**:
I have divided the change into 4 commits
1. Code for the Volume Plugin
2. Configuration Files, Examples, Validation Files
3. Godeps
4. Generated code and Documentation

```release-note
Portworx Volume Plugin added enabling [Portworx](http://www.portworx.com) to be used as a storage provider for Kubernetes clusters. Portworx pools your servers capacity and turns your servers or cloud instances into converged, highly available compute and storage nodes.
```

Automatic merge from submit-queue

fix upgrades

Automatic merge from submit-queue

remove trusty GCE kube-up.sh

Asked on the mailing list. No one objected. Lot's of people were in favor.

cc @roberthbailey 


```release-note
Remove support for trusty in GCE kube-up.
```

Add tracking on resource.Builder if a "named" item is requested (from
file, stream, url, or resource args) and use that in `get` to accurately
determine whether to filter resources. Add tests.

Automatic merge from submit-queue

Correct CIDR range for kubemark

- Add a new type PortworxVolumeSource
- Implement the kubernetes volume plugin for Portworx Volumes under pkg/volume/portworx
- The Portworx Volume Driver uses the libopenstorage/openstorage specifications and apis for volume operations.

Changes for k8s configuration and examples for portworx volumes.

- Add PortworxVolume hooks in kubectl, kube-controller-manager and validation.
- Add a README for PortworxVolume usage as PVs, PVCs and StorageClass.
- Add example spec files

Handle code review comments.

- Modified READMEs to incorporate to suggestions.
- Add a test for ReadWriteMany access mode.
- Use util.UnmountPath in TearDown.
- Add ReadOnly flag to PortworxVolumeSource
- Use hostname:port instead of unix sockets
- Delete the mount dir in TearDown.
- Fix link issue in persistentvolumes README
- In unit test check for mountpath after Setup is done.
- Add PVC Claim Name as a Portworx Volume Label

Generated code and documentation.
- Updated swagger spec
- Updated api-reference docs
- Updated generated code under pkg/api/v1

Godeps update for Portworx Volume Driver
- Adds github.com/libopenstorage/openstorage
- Adds go.pedge.io/pb/go/google/protobuf
- Updates Godep Licenses

Remove rubbish file

[Federation] Remove federat{ed,ion} prefixes from e2e file names since they are all now scoped under the e2e_federation package.

Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)

Use chroot for containerized mounts

This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.

Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)

Fix azure file secret reference

Follow up to https://github.com/kubernetes/kubernetes/pull/41957

Fixes nil dereference getting secret name from AzureFile volume source.

Adds unit tests to make sure all secret references are extracted correctly, and adds reflective tests to help catch drift if new secret references are added to the pod spec

Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)

add aggregation integration test

Wires up an integration test which runs a full kube-apiserver, the wardle server, and the kube-aggregator and creates the APIservice object for the wardle server.  Without services and DNS the aggregator doesn't proxy, but it does ensure we don't have an obvious panic or bring up failure.

@sttts @ncdc

Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)

client-gen: create private registry in fake clientset

This cuts of the laster `k8s.io/kubernetes/pkg/api.{Registry+Scheme+Codecs}` dependency from the clientsets. This enables clientset generation for packages that must not have a dependency onto kubernetes itself.

@deads2k there is more than the namespace checking we discussed: the RESTMapper built from the registry. This introduces a private registry. I try get that out from the normal versioned client as much as possible. I would even like to remove this private registry some day, at best remove all registry code from the client. But that's for another day...

Automatic merge from submit-queue

kubectl describe: show annotations

Fixes https://github.com/kubernetes/kubernetes/issues/23380

Automatic merge from submit-queue (batch tested with PRs 40746, 41699, 42108, 42174, 42093)

Avoid fake node names in user info

Node usernames should follow the format `system:node:<node-name>`,
but if we don't know the node name, it's worse to put a fake one in.

In the future, we plan to have a dedicated node authorizer, which would
start rejecting requests from a user with a bogus node name like this.

The right approach is to either mint correct credentials per node, or use node bootstrapping so it requests a correct client certificate itself.

Automatic merge from submit-queue (batch tested with PRs 40746, 41699, 42108, 42174, 42093)

test: move deployment e2e test out of the flaky suite

Moves the deployment test that was marked as flaky in https://github.com/kubernetes/kubernetes/pull/41257 out of the flaky suite.

Haven't seen this flake for some time now:
https://k8s-testgrid.appspot.com/google-gce#gci-gce-flaky
https://k8s-testgrid.appspot.com/google-gce#gce-flaky

Also https://github.com/kubernetes/kubernetes/pull/41510 is merged.

Closes https://github.com/kubernetes/kubernetes/issues/39785 

@spxtr @janetkuo

Automatic merge from submit-queue (batch tested with PRs 40746, 41699, 42108, 42174, 42093)

Switch kube-proxy to informers & save 2/3 of cpu & memory of non-iptables related code.

Fix #42000

This PR should be no-op from the behavior perspective.
It is changing KubeProxy to use standard "informer" framework instead of combination of reflector + undelta store.

This is significantly reducing CPU usage of kube-proxy and number of memory allocations.
Previously, on every endpoints/service update, we were copying __all__ endpoints/services at least 3 times, now it is once (which should also be removed in the future).

In Kubemark-500, hollow-proxies were processing backlog from load test for an hour after the test was finishing. With this change, it is keeping up with the load.

@thockin @ncdc @derekwaynecarr

Automatic merge from submit-queue (batch tested with PRs 40746, 41699, 42108, 42174, 42093)

Output result of apply operation

Fixes #41690 

Plumbs the resulting object from patch operations back to the top level so it can be output when printing

Automatic merge from submit-queue

numeric ordering of kubectl outputs

**What this PR does / why we need it**:
Instead of having kubectl listing the pods in a alphabetical way:
foobar-1-build
foobar-10-build
foobar-2-build
foobar-3-build
With the parameter --sort-by '{.metadata.name}' it now gives:
foobar-1-build
foobar-2-build
foobar-3-build
foobar-10-build

**Which issue this PR fixes**
https://github.com/openshift/origin/issues/7229

**Special notes for your reviewer**:
I have followed the dependencies requirements from https://github.com/kubernetes/community/blob/master/contributors/devel/godep.md

**Release note**:

```release-note
Import a natural sorting library and use it in the sorting printer.
```