Automatic merge from submit-queue

Only set sysctls in infra container in dockershim

Dockershim counterpart of https://github.com/kubernetes/kubernetes/pull/32383.

Fix typos

Automatic merge from submit-queue

Update verify-openapi-spec script to check for extra generated spec

hack/verify-openapi-spec.sh only check for existing spec changes. If for some reason (here most probably I forgot to delete a file in api/openapi-spec folder in #35388 after a rebase) there is an old spec exists in the spec folder, it won't panic but it should. This resulted in an unused out of date v1.spec file in the api/openapi-spec folder that this PR also removes.

Automatic merge from submit-queue

CRI: Handle empty container name in dockershim.

Fixes https://github.com/kubernetes/kubernetes/issues/35924.

Dead container may have no name, we should handle this properly.

@yujuhong @bprashanth

Automatic merge from submit-queue

make ./pkg/client/listers compile

currently compilation is broken

```
$ go install ./pkg/client/listers/...
# k8s.io/kubernetes/pkg/client/listers/apps/v1alpha1
pkg/client/listers/apps/v1alpha1/zz_generated.statefulset.go:89: undefined: apps in apps.Resource
# k8s.io/kubernetes/pkg/client/listers/autoscaling/v1
pkg/client/listers/autoscaling/v1/zz_generated.horizontalpodautoscaler.go:89: undefined: autoscaling in autoscaling.Resource
# k8s.io/kubernetes/pkg/client/listers/batch/v2alpha1
pkg/client/listers/batch/v2alpha1/zz_generated.job.go:89: undefined: batch in batch.Resource
pkg/client/listers/batch/v2alpha1/zz_generated.scheduledjob.go:89: undefined: batch in batch.Resource
# k8s.io/kubernetes/pkg/client/listers/authentication/v1beta1
pkg/client/listers/authentication/v1beta1/zz_generated.tokenreview.go:63: undefined: authentication in authentication.Resource
# k8s.io/kubernetes/pkg/client/listers/batch/v1
pkg/client/listers/batch/v1/zz_generated.job.go:89: undefined: batch in batch.Resource
# k8s.io/kubernetes/pkg/client/listers/authorization/v1beta1
pkg/client/listers/authorization/v1beta1/zz_generated.localsubjectaccessreview.go:89: undefined: authorization in authorization.Resource
pkg/client/listers/authorization/v1beta1/zz_generated.selfsubjectaccessreview.go:63: undefined: authorization in authorization.Resource
pkg/client/listers/authorization/v1beta1/zz_generated.subjectaccessreview.go:63: undefined: authorization in authorization.Resource
# k8s.io/kubernetes/pkg/client/listers/certificates/v1alpha1
pkg/client/listers/certificates/v1alpha1/zz_generated.certificatesigningrequest.go:63: undefined: certificates in certificates.Resource
# k8s.io/kubernetes/pkg/client/listers/policy/v1alpha1
pkg/client/listers/policy/v1alpha1/zz_generated.poddisruptionbudget.go:89: undefined: policy in policy.Resource
# k8s.io/kubernetes/pkg/client/listers/core/v1
pkg/client/listers/core/v1/zz_generated.componentstatus.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.configmap.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.endpoints.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.event.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.limitrange.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.namespace.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.node.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.persistentvolume.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.persistentvolumeclaim.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.pod.go:89: undefined: api
pkg/client/listers/core/v1/zz_generated.pod.go:89: too many errors
# k8s.io/kubernetes/pkg/client/listers/imagepolicy/v1alpha1
pkg/client/listers/imagepolicy/v1alpha1/zz_generated.imagereview.go:63: undefined: imagepolicy in imagepolicy.Resource
# k8s.io/kubernetes/pkg/client/listers/rbac/v1alpha1
pkg/client/listers/rbac/v1alpha1/zz_generated.clusterrole.go:63: undefined: rbac in rbac.Resource
pkg/client/listers/rbac/v1alpha1/zz_generated.clusterrolebinding.go:63: undefined: rbac in rbac.Resource
pkg/client/listers/rbac/v1alpha1/zz_generated.role.go:89: undefined: rbac in rbac.Resource
pkg/client/listers/rbac/v1alpha1/zz_generated.rolebinding.go:89: undefined: rbac in rbac.Resource
# k8s.io/kubernetes/pkg/client/listers/storage/v1beta1
pkg/client/listers/storage/v1beta1/zz_generated.storageclass.go:63: undefined: storage in storage.Resource
# k8s.io/kubernetes/pkg/client/listers/extensions/v1beta1
pkg/client/listers/extensions/v1beta1/zz_generated.daemonset.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.deployment.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.ingress.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.job.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.podsecuritypolicy.go:63: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.replicaset.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.scale.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.thirdpartyresource.go:63: undefined: extensions in extensions.Resource
```
cc @ncdc @caesarxuchao

Automatic merge from submit-queue

Disable gci-mounter in cri node e2e tests

gci-mounter is still being validated and there are known issues. Do not enable it
for cri tests for now.

Automatic merge from submit-queue

CRI: Add kuberuntime container logs

Based on https://github.com/kubernetes/kubernetes/pull/34858.

The first 2 commits are from #34858. And the last 2 commits are new.

This PR added kuberuntime container logs support and add unit test for it.

I've tested all the functions manually, and I'll send another PR to write a node e2e test for container log.

**_Notice: current implementation doesn't support log rotation**_, which means that:
- It will not retrieve logs in rotated log file.
- If log rotation happens when following the log:
  - If the rotation is using create mode, we'll still follow the old file.
  - If the rotation is using copytruncate, we'll be reading at the original position and get nothing.

To solve these issues, kubelet needs to rotate the log itself, or at least kubelet should be able to control the the behavior of log rotator. These are doable but out of the scope of 1.5 and will be addressed in future release.

@yujuhong @feiskyer @yifan-gu 
/cc @kubernetes/sig-node

Automatic merge from submit-queue

Node controller to not force delete pods

Fixes https://github.com/kubernetes/kubernetes/issues/35145

- [x] e2e tests to test Petset, RC, Job.
- [x] Remove and cover other locations where we force-delete pods within the NodeController.

**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
Node controller no longer force-deletes pods from the api-server.

* For StatefulSet (previously PetSet), this change means creation of replacement pods is blocked until old pods are definitely not running (indicated either by the kubelet returning from partitioned state, or deletion of the Node object, or deletion of the instance in the cloud provider, or force deletion of the pod from the api-server). This has the desirable outcome of "fencing" to prevent "split brain" scenarios.
* For all other existing controllers except StatefulSet , this has no effect on the ability of the controller to replace pods because the controllers do not reuse pod names (they use generate-name).
* User-written controllers that reuse names of pod objects should evaluate this change.
```

Automatic merge from submit-queue

AWS: recognize us-east-2 region

Automatic merge from submit-queue

CRI: Rename container/sandbox states

The enum constants are not namespaced. The shorter, unspecifc names are likely
to cause naming conflicts in the future.

Also replace "SandBox" with "Sandbox" in the API for consistency.

/cc @kubernetes/sig-node

gci-mounter is still being validated and there are known issues. Do not enable it
for cri tests for now.

Automatic merge from submit-queue

support Azure disk dynamic provisioning

azure disk dynamic provisioning

A screen shot 

``` console
$ kubectl create -f examples/experimental/persistent-volume-provisioning/azure-dd.yaml
storageclass "slow" created
$ kubectl create -f examples/experimental/persistent-volume-provisioning/claim1.json
persistentvolumeclaim "claim1" created
$ kubectl describe pvc
Name:       claim1
Namespace:  default
Status:     Bound
Volume:     pvc-de7150d1-6a37-11e6-aec9-000d3a12e034
Labels:     <none>
Capacity:   3Gi
Access Modes:   RWO
$ kubectl create -f pod.yaml
replicationcontroller "nfs-server" created
$ kubectl describe pod
Name:       nfs-server-b9w6x
Namespace:  default
Node:       rootfs-dev/172.24.0.4
Start Time: Wed, 24 Aug 2016 19:46:21 +0000
Labels:     role=nfs-server
Status:     Running
IP:     172.17.0.2
Controllers:    ReplicationController/nfs-server
Containers:
  nfs-server:
    Container ID:   docker://be6f8c0e26dc896d4c53ef0d21c9414982f0b39a10facd6b93a255f9e1c3806c
    Image:      nginx
    Image ID:       docker://bfdd4ced794ed276a28cf56b233ea58dec544e9ca329d796cf30b8bcf6d39b3f
    Port:       
    State:      Running
      Started:      Wed, 24 Aug 2016 19:49:19 +0000
    Ready:      True
    Restart Count:  0
    Volume Mounts:
      /exports from mypvc (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9o0fj (ro)
    Environment Variables:  <none>
Conditions:
  Type      Status
  Initialized   True 
  Ready     True 
  PodScheduled  True 
Volumes:
  mypvc:
    Type:   PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  claim1
    ReadOnly:   false
  default-token-9o0fj:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-9o0fj
QoS Class:  BestEffort
Tolerations:    <none>
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath           Type        Reason      Message
  --------- --------    -----   ----            -------------           --------    ------      -------
  11m       11m     1   {default-scheduler }                    Normal      Scheduled   Successfully assigned nfs-server-b9w6x to rootfs-dev
  9m        9m      1   {kubelet rootfs-dev}                    Warning     FailedMount Unable to mount volumes for pod "nfs-server-b9w6x_default(6eb7fd98-6a33-11e6-aec9-000d3a12e034)": timeout expired waiting for volumes to attach/mount for pod "nfs-server-b9w6x"/"default". list of unattached/unmounted volumes=[mypvc]
  9m        9m      1   {kubelet rootfs-dev}                    Warning     FailedSync  Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "nfs-server-b9w6x"/"default". list of unattached/unmounted volumes=[mypvc]
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Pulling     pulling image "nginx"
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Pulled      Successfully pulled image "nginx"
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Created     Created container with docker id be6f8c0e26dc
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Started     Started container with docker id be6f8c0e26dc

```

@colemickens @brendandburns

Automatic merge from submit-queue

Add FeatureGates field to KubeletConfiguration

This threads the `--feature-gates` flag through the `KubeletConfiguration` object and also allows setting feature gates via dynamic Kubelet configuration. 

/cc @jlowdermilk

Automatic merge from submit-queue

e2e.go/kops: Bump timeout to 20m, fix KUBERNETES_PROVIDER

**What this PR does / why we need it**: I don't have a ton of proof, but I think https://k8s-testgrid.appspot.com/google-aws#kops-aws-updown builds 4045-4047 are just AWS and DNS slowness.

In addition, my original PR was meant to change `KUBERNETES_PROVIDER` based on `KUBERNETES_CONFORMANCE_PROVIDER`.

Automatic merge from submit-queue

kubeadm preflight checks: Warn user if connections to API or Discovery are going to be over proxy

**What this PR does / why we need it**: Continuing discussion from PR #35044, new version will provide warning if kubeadm run in environment where http connections would go over proxy.
Most of the time, it is not expected behaviour and leads to situations like in #34695

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #34695

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

kubeadm during initialization of master and slave nodes need to make
several API calls directly to the node where it is running or master.
In environments with http/https proxies, user might accidentally
have configuration where connections to API would go over proxy instead
of directly.

User can re-run kubeadm with corrected NO_PROXY variable. Example:

  $ NO_PROXY=* kubeadm join ...

Along the way: Fix ginkgo-e2e.sh. This change got dropped in the
original PR, but it was meant to allow a conformance-style
kubectl-auth test and still have a legit cloud provider.

Revert "fixed some issues with kubectl set resources"

Automatic merge from submit-queue

Disruption e2e: wait for running pods in the table test, too

**What this PR does / why we need it**: Makes the `DisruptionController` tests less flaky.
**Which issue this PR fixes**: fixes #34032

The enum constants are not namespaced. The shorter, unspecifc names are likely
to cause naming conflicts in the future.

Also replace "SandBox" with "Sandbox" in the API.

[Federation][(Un)join-00] Implement federation/cluster resource generator.

Automatic merge from submit-queue

Remove an unnecessary log line in kubelet

Automatic merge from submit-queue

e2e: Fix GetReadySchedulableNodesOrDie for taints

**What this PR does / why we need it**:

This changes framework.GetReadySchedulableNodesOrDie and
framework.GetMasterAndWorkerNodesOrDie so that nodes that can't take a
generic fake pod due to a taint/toleration mismatch aren't returned.

This is a rehash of #35210, but pulls in the scheduler code.

**Which issue this PR fixes**: c.f. #35210

**Special notes for your reviewer**: I think it's gross that we keep having to manually compute this in e2es. Maybe we need a bug for that?

Automatic merge from submit-queue

Migrate Network Logic into runtime II

- [x] Probing
- [x] Bootstrap CIDR
- [x] Setup/Teardown
- [ ] ~~Hostports~~
- [ ] ~~BWShaping~~

https://github.com/kubernetes/kubernetes/pull/34780

Automatic merge from submit-queue

Eviction manager evicts based on inode consumption

Fixes: #32526 Integrate Cadvisor per-container inode stats into the summary api.  Make the eviction manager act based on inode consumption to evict pods using the most inodes.

This PR is pending on a cadvisor godeps update which will be included in PR #35136

This changes framework.GetReadySchedulableNodesOrDie and
framework.GetMasterAndWorkerNodesOrDie so that nodes that can't take a
generic fake pod due to a taint/toleration mismatch aren't returned.

This is a rehash of #35210, but pulls in the scheduler code.

Automatic merge from submit-queue

Add tooling to generate listers

Add lister-gen tool to auto-generate listers. So far this PR only demonstrates replacing the manually-written `StoreToLimitRangeLister` with the generated `LimitRangeLister`, as it's a small and easy swap.

cc @deads2k @liggitt @sttts @nikhiljindal @lavalamp @smarterclayton @derekwaynecarr  @kubernetes/sig-api-machinery @kubernetes/rh-cluster-infra

Automatic merge from submit-queue

Only set sysctls for infra containers

We did set the sysctls for each container in a pod. This opens up a way to set un-whitelisted sysctls during upgrade from v1.3:
- set annotation in v1.3 with an un-whitelisted sysctl. Set restartPolicy=Always
- upgrade cluster to v1.4
- kill container process
- un-whitelisted sysctl is set on restart of the killed container.

kubeadm during initialization of master and slave nodes need to make
several API calls directly to the node where it is running or master.
In environments with http/https proxies, user might accidentally
have configuration where connections to API would go over proxy instead
of directly.

User can re-run kubeadm with corrected NO_PROXY variable. Example:

  $ NO_PROXY=* kubeadm join ...

Fix density test in large clusters

Automatic merge from submit-queue

kubeadm: lower default component logging level

v=4 is likely to causer perf issues and v=2 is the default in most other setups including scale e2e. ref https://github.com/kubernetes/kubernetes/issues/26637#issuecomment-256859739