Automatic merge from submit-queue

kubectl commands must not use the factory out of Run

Fixes https://github.com/kubernetes/kubernetes/issues/35426

@smarterclayton PTAL

Automatic merge from submit-queue

Require PV provisioner secrets to match type

In 1.5, PV provisioners are allowing targeting namespaced secrets via storageclass params. This adds a requirement that those secrets' type match the volume provisioner plugin name, to prevent targeting and extraction of arbitrary secrets

Helps limit secret targeting issues mentioned in https://github.com/kubernetes/kubernetes/issues/34822

Automatic merge from submit-queue

Add test image for networking related tests

This test image is to support the e2e test for #32551

Automatic merge from submit-queue

Add "PrintErrorWithCauses" cmdutil helper

**Release note**:
```release-note
NONE
```

This patch adds a new helper function to `cmd/util/helpers.go` that
handles errors containing collections of causes and prints each cause in
a separate newline.

Automatic merge from submit-queue

Use osInterface for Create

A small fix of using osInterface instead of os for `Create()`.

cc/ @Random-Liu

Skip scheduleJob e2e test if it's not enabled on server

Automatic merge from submit-queue

Verify and update client-go staging area for every PR

We need to keep the staging area up-to-date to prevent PRs from breaking client-go.

It's marked as "WIP" because we need to decide the [versioning strategy](https://github.com/kubernetes/client-go/issues/9) for client-go first. This PR contains breaking changes for client-go.

This is blocking #29934 and potentially #34441

cc @kubernetes/sig-api-machinery

Automatic merge from submit-queue

Let release_1_5 clientset include multiple versions of a group

Fix #35237 

This PR make versioned clientset to include multiple versions of a group. Currently only `batch` has `v1` and `v2alpha1`. The clientset interface now looks like:
```go
	BatchV2alpha1() v2alpha1batch.BatchV2alpha1Interface
	BatchV1() v1batch.BatchV1Interface
	// Deprecated: please explicitly pick a version if possible.
	Batch() v1batch.BatchV1Interface
```

Commit "update client-gen to say internalversion rather than unversioned" fixes https://github.com/kubernetes/kubernetes/issues/24481. 


cc @kubernetes/sig-api-machinery @soltysh @deads2k @nikhiljindal 



```release-note
release_1_5 clientset supports multiple versions of a group.
```

Automatic merge from submit-queue

Make overlapping deployments deletable

@kubernetes/deployment ptal

Fixes https://github.com/kubernetes/kubernetes/issues/34466 by 1) not adding the overlapping annotation in the working deployment, 2) updates observedGeneration for overlapping deployments, and 3) updates the kubectl deployment reaper to do non-cascading deletion for deployments with the overlapping annotation.

fix test/e2e/disruption after running copy.sh

disable verify-staging-client-go.sh until 1.5 code freeze

update client-gen to use the term "internalversion" rather than "unversioned";
leave internal one unqualified;
cleanup client-gen

Automatic merge from submit-queue

Convert - to _ for protobuf package names

Convert - to _ for protobuf package names to allow protobuf code generation
support for go packages that have - in their names.

@smarterclayton @deads2k @liggitt @sttts @lavalamp @nikhiljindal @kubernetes/sig-api-machinery

Automatic merge from submit-queue

Add boilerplate to `kubectl completion bash`

**What this PR does / why we need it**: 

Small refactor to make kubectl bash and zsh completion share
boilerplate.  Previously the boilerplate was not included in the bash
script.

Automatic merge from submit-queue

support editing before creating resource

Support `kubectl create -f config.yaml --edit`
Support editing before creating resource from files, urls and stdin.
The behavior is similar to `kubectl edit`
It won't create anything when edit make no change.

partial: #18064

Based on: #33686 and #33973

```release-note
Support editing before creating resource from files, urls and stdin, e.g. `kubectl create -f config.yaml --edit`
It won't create anything when edit make no change.
```

Automatic merge from submit-queue

convert SA controller to shared informers

convert the SA controller to shared informer + workqueue.

I think one of @derekwaynecarr @ncdc or @liggitt

Automatic merge from submit-queue

Implement streaming CRI methods in dockershim

*NOTE: Temporarily includes commit from https://github.com/kubernetes/kubernetes/pull/35330 - only review the second commit.*

Builds on https://github.com/kubernetes/kubernetes/pull/35330, using the library to implement the streaming methods in various CRI shims.

This does not actually wire up the new streaming methods in the kubelet (that will be my next PR). Once the new methods are wired up, I will delete the `Legacy{Exec,Attach,PortForward}` methods.

/cc @kubernetes/sig-node @feiskyer

Automatic merge from submit-queue

Removed 1.3 clientset

needed for #35782

Automatic merge from submit-queue

allow authentication through a front-proxy

This allows a front proxy to set a request header and have that be a valid `user.Info` in the authentication chain.  To secure this power, a client certificate may be used to confirm the identity of the front proxy

@kubernetes/sig-auth fyi
@erictune per-request
@liggitt you wrote the openshift one, ptal.

Automatic merge from submit-queue

Fix federation-apiserver authentication e2e tests

**What this PR does / why we need it**: Fixes couple of federation e2e tests

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # partly fixes #35251

Automatic merge from submit-queue

Implement package `triple` with utilities to generate certificate-key pairs for CA, server and clients.

Please review only the last commit here. This is based on PRs #35592 which will be reviewed independently.

Design Doc: PR #34484

cc @kubernetes/sig-cluster-federation @quinton-hoole @mwielgus

Automatic merge from submit-queue

Rename TEST_ETCD_VERSION to ETCD_VERSION

Ref #20504

Automatic merge from submit-queue

Update Godeps etcd to 3.0.13

Ref #20504

Automatic merge from submit-queue

Clean up static-ip in e2e

I dropped this line: https://github.com/kubernetes/kubernetes/pull/35331/files#diff-5997d488011ead8354bdddfceaf1515dL595, in a previous refactor https://github.com/kubernetes/kubernetes/pull/35331, so we started leaking static ips. This commit adds it back in, along with some debug info if it happens again. Marking p2 since test is currently leaking.

Automatic merge from submit-queue

Simplify negotiation in server in preparation for multi version support

This is a pre-factor for #33900 to simplify runtime.NegotiatedSerializer, tighten up a few abstractions that may break when clients can request different client versions, and pave the way for better negotiation.

View this as pure simplification.

Automatic merge from submit-queue

Fixing e2e tests which rely on network disruptions

**What this PR does / why we need it**: It fixes e2e tests  on network disruptions

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/kubernetes/issues/27324, https://github.com/kubernetes/kubernetes/issues/35293

https://github.com/kubernetes/kubernetes/pull/33655 changed the kubelet's `--api-server` flag to use the external IP address of the APIServer in GCE. Hence, the iptables rules were failing previously. We now return the external IP for `getMaster()`. This fix is required in order to write e2e tests which test behavior in the face of network disruptions.

/cc @bprashanth @dchen1107 @kubernetes/sig-apps

Automatic merge from submit-queue

On GCI, cleanup kubelet startup

-->
```release-note
* Avoid overriding system and kubelet cgroups on GCI
* Make the kubectl from k8s release the default on GCI
```

cc @kubernetes/sig-node @mtaufen

Automatic merge from submit-queue

Add a retry when reading a file content from a container

To avoid temporal failure in reading the file content, add a retry
process in function verifyPDContentsViaContainer

Automatic merge from submit-queue

Fix cadvisor_unsupported and the crossbuild

Resolves a bug in the `cadvisor_unsupported.go` code.
Fixes https://github.com/kubernetes/kubernetes/issues/35735

Introduced by: https://github.com/kubernetes/kubernetes/pull/35136
We should consider to cherrypick this as #35136 also was cherrypicked

cc @kubernetes/sig-testing @vishh @dashpole @jessfraz

```release-note
Fix cadvisor_unsupported and the crossbuild
```

Automatic merge from submit-queue

[PHASE 1] Opaque integer resource accounting.

## [PHASE 1] Opaque integer resource accounting.

This change provides a simple way to advertise some amount of arbitrary countable resource for a node in a Kubernetes cluster. Users can consume these resources by including them in pod specs, and the scheduler takes them into account when placing pods on nodes. See the example at the bottom of the PR description for more info.

Summary of changes:

- Defines opaque integer resources as any resource with prefix `pod.alpha.kubernetes.io/opaque-int-resource-`.
- Prevent kubelet from overwriting capacity.
- Handle opaque resources in scheduler.
- Validate integer-ness of opaque int quantities in API server.
- Tests for above.

Feature issue: https://github.com/kubernetes/features/issues/76

Design: http://goo.gl/IoKYP1

Issues:

kubernetes/kubernetes#28312
kubernetes/kubernetes#19082

Related:

kubernetes/kubernetes#19080

CC @davidopp @timothysc @balajismaniam 

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
Added support for accounting opaque integer resources.

Allows cluster operators to advertise new node-level resources that would be
otherwise unknown to Kubernetes. Users can consume these resources in pod
specs just like CPU and memory. The scheduler takes care of the resource
accounting so that no more than the available amount is simultaneously
allocated to pods.
```

## Usage example

```sh
$ echo '[{"op": "add", "path": "pod.alpha.kubernetes.io~1opaque-int-resource-bananas", "value": "555"}]' | \
> http PATCH http://localhost:8080/api/v1/nodes/localhost.localdomain/status \
> Content-Type:application/json-patch+json
```

```http
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 11 Aug 2016 16:44:55 GMT
Transfer-Encoding: chunked

{
    "apiVersion": "v1",
    "kind": "Node",
    "metadata": {
        "annotations": {
            "volumes.kubernetes.io/controller-managed-attach-detach": "true"
        },
        "creationTimestamp": "2016-07-12T04:07:43Z",
        "labels": {
            "beta.kubernetes.io/arch": "amd64",
            "beta.kubernetes.io/os": "linux",
            "kubernetes.io/hostname": "localhost.localdomain"
        },
        "name": "localhost.localdomain",
        "resourceVersion": "12837",
        "selfLink": "/api/v1/nodes/localhost.localdomain/status",
        "uid": "2ee9ea1c-47e6-11e6-9fb4-525400659b2e"
    },
    "spec": {
        "externalID": "localhost.localdomain"
    },
    "status": {
        "addresses": [
            {
                "address": "10.0.2.15",
                "type": "LegacyHostIP"
            },
            {
                "address": "10.0.2.15",
                "type": "InternalIP"
            }
        ],
        "allocatable": {
            "alpha.kubernetes.io/nvidia-gpu": "0",
            "cpu": "2",
            "memory": "8175808Ki",
            "pods": "110"
        },
        "capacity": {
            "alpha.kubernetes.io/nvidia-gpu": "0",
            "pod.alpha.kubernetes.io/opaque-int-resource-bananas": "555",
            "cpu": "2",
            "memory": "8175808Ki",
            "pods": "110"
        },
        "conditions": [
            {
                "lastHeartbeatTime": "2016-08-11T16:44:47Z",
                "lastTransitionTime": "2016-07-12T04:07:43Z",
                "message": "kubelet has sufficient disk space available",
                "reason": "KubeletHasSufficientDisk",
                "status": "False",
                "type": "OutOfDisk"
            },
            {
                "lastHeartbeatTime": "2016-08-11T16:44:47Z",
                "lastTransitionTime": "2016-07-12T04:07:43Z",
                "message": "kubelet has sufficient memory available",
                "reason": "KubeletHasSufficientMemory",
                "status": "False",
                "type": "MemoryPressure"
            },
            {
                "lastHeartbeatTime": "2016-08-11T16:44:47Z",
                "lastTransitionTime": "2016-08-10T06:27:11Z",
                "message": "kubelet is posting ready status",
                "reason": "KubeletReady",
                "status": "True",
                "type": "Ready"
            },
            {
                "lastHeartbeatTime": "2016-08-11T16:44:47Z",
                "lastTransitionTime": "2016-08-10T06:27:01Z",
                "message": "kubelet has no disk pressure",
                "reason": "KubeletHasNoDiskPressure",
                "status": "False",
                "type": "DiskPressure"
            }
        ],
        "daemonEndpoints": {
            "kubeletEndpoint": {
                "Port": 10250
            }
        },
        "images": [],
        "nodeInfo": {
            "architecture": "amd64",
            "bootID": "1f7e95ca-a4c2-490e-8ca2-6621ae1eb5f0",
            "containerRuntimeVersion": "docker://1.10.3",
            "kernelVersion": "4.5.7-202.fc23.x86_64",
            "kubeProxyVersion": "v1.3.0-alpha.4.4285+7e4b86c96110d3-dirty",
            "kubeletVersion": "v1.3.0-alpha.4.4285+7e4b86c96110d3-dirty",
            "machineID": "cac4063395254bc89d06af5d05322453",
            "operatingSystem": "linux",
            "osImage": "Fedora 23 (Cloud Edition)",
            "systemUUID": "D6EE0782-5DEB-4465-B35D-E54190C5EE96"
        }
    }
}
```

After patching, the kubelet's next sync fills in allocatable:

```
$ kubectl get node localhost.localdomain -o json | jq .status.allocatable
```

```json
{
  "alpha.kubernetes.io/nvidia-gpu": "0",
  "pod.alpha.kubernetes.io/opaque-int-resource-bananas": "555",
  "cpu": "2",
  "memory": "8175808Ki",
  "pods": "110"
}
```

Create two pods, one that needs a single banana and another that needs a truck load:

```
$ kubectl create -f chimp.yaml
$ kubectl create -f superchimp.yaml
```

Inspect the scheduler result and pod status:

```
$ kubectl describe pods chimp
Name:           chimp
Namespace:      default
Node:           localhost.localdomain/10.0.2.15
Start Time:     Thu, 11 Aug 2016 19:58:46 +0000
Labels:         <none>
Status:         Running
IP:             172.17.0.2
Controllers:    <none>
Containers:
  nginx:
    Container ID:       docker://46ff268f2f9217c59cc49f97cc4f0f085d5ac0e251f508cc08938601117c0cec
    Image:              nginx:1.10
    Image ID:           docker://sha256:82e97a2b0390a20107ab1310dea17f539ff6034438099384998fd91fc540b128
    Port:               80/TCP
    Limits:
      cpu:                                      500m
      memory:                                   64Mi
      pod.alpha.kubernetes.io/opaque-int-resource-bananas:   3
    Requests:
      cpu:                                      250m
      memory:                                   32Mi
      pod.alpha.kubernetes.io/opaque-int-resource-bananas:   1
    State:                                      Running
      Started:                                  Thu, 11 Aug 2016 19:58:51 +0000
    Ready:                                      True
    Restart Count:                              0
    Volume Mounts:                              <none>
    Environment Variables:                      <none>
Conditions:
  Type          Status
  Initialized   True 
  Ready         True 
  PodScheduled  True 
No volumes.
QoS Class:      Burstable
Events:
  FirstSeen     LastSeen        Count   From                            SubobjectPath           Type            Reason                  Message
  ---------     --------        -----   ----                            -------------           --------        ------                  -------
  9m            9m              1       {default-scheduler }                                    Normal          Scheduled               Successfully assigned chimp to localhost.localdomain
  9m            9m              2       {kubelet localhost.localdomain}                         Warning         MissingClusterDNS       kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
  9m            9m              1       {kubelet localhost.localdomain} spec.containers{nginx}  Normal          Pulled                  Container image "nginx:1.10" already present on machine
  9m            9m              1       {kubelet localhost.localdomain} spec.containers{nginx}  Normal          Created                 Created container with docker id 46ff268f2f92
  9m            9m              1       {kubelet localhost.localdomain} spec.containers{nginx}  Normal          Started                 Started container with docker id 46ff268f2f92
```

```
$ kubectl describe pods superchimp
Name:           superchimp
Namespace:      default
Node:           /
Labels:         <none>
Status:         Pending
IP:
Controllers:    <none>
Containers:
  nginx:
    Image:      nginx:1.10
    Port:       80/TCP
    Requests:
      cpu:                                      250m
      memory:                                   32Mi
      pod.alpha.kubernetes.io/opaque-int-resource-bananas:   10Ki
    Volume Mounts:                              <none>
    Environment Variables:                      <none>
Conditions:
  Type          Status
  PodScheduled  False 
No volumes.
QoS Class:      Burstable
Events:
  FirstSeen     LastSeen        Count   From                    SubobjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                    -------------   --------        ------                  -------
  3m            1s              15      {default-scheduler }                    Warning         FailedScheduling        pod (superchimp) failed to fit in any node
fit failure on node (localhost.localdomain): Insufficient pod.alpha.kubernetes.io/opaque-int-resource-bananas
```

Increase sys container usageBytes upper bound

Automatic merge from submit-queue

Add netbase package to gci mounter container image

cc @rootfs @jingxu97

Automatic merge from submit-queue

Add sync state loop in master's volume reconciler

At master volume reconciler, the information about which volumes are
attached to nodes is cached in actual state of world. However, this
information might be out of date in case that node is terminated (volume
is detached automatically). In this situation, reconciler assume volume
is still attached and will not issue attach operation when node comes
back. Pods created on those nodes will fail to mount.
This PR adds the logic to periodically sync up the truth for attached
volumes kept in
the actual state cache. If the volume is no longer attached to the node,
the actual state will be updated to reflect the truth. In turn,
reconciler will take actions if needed.
To avoid issuing many concurrent operations on cloud provider, this PR
tries to add batch operation to check whether a list of volumes are
attached to the node instead of one request per volume.

Bump GCE debian image to container-vm-v20161025 (CVE-2016-5195 Dirty…

Automatic merge from submit-queue

Improve update_owners.py username detection and error message.

Fixes the root cause of #35808.

Automatic merge from submit-queue

GC pod ips

Finally managed to write a *failing* test. 
Supersedes https://github.com/kubernetes/kubernetes/pull/34373

```release-note
GC pod ips
```