Automatic merge from submit-queue

CRI: Add kuberuntime container logs

Based on https://github.com/kubernetes/kubernetes/pull/34858.

The first 2 commits are from #34858. And the last 2 commits are new.

This PR added kuberuntime container logs support and add unit test for it.

I've tested all the functions manually, and I'll send another PR to write a node e2e test for container log.

**_Notice: current implementation doesn't support log rotation**_, which means that:
- It will not retrieve logs in rotated log file.
- If log rotation happens when following the log:
  - If the rotation is using create mode, we'll still follow the old file.
  - If the rotation is using copytruncate, we'll be reading at the original position and get nothing.

To solve these issues, kubelet needs to rotate the log itself, or at least kubelet should be able to control the the behavior of log rotator. These are doable but out of the scope of 1.5 and will be addressed in future release.

@yujuhong @feiskyer @yifan-gu 
/cc @kubernetes/sig-node

Automatic merge from submit-queue

Node controller to not force delete pods

Fixes https://github.com/kubernetes/kubernetes/issues/35145

- [x] e2e tests to test Petset, RC, Job.
- [x] Remove and cover other locations where we force-delete pods within the NodeController.

**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
Node controller no longer force-deletes pods from the api-server.

* For StatefulSet (previously PetSet), this change means creation of replacement pods is blocked until old pods are definitely not running (indicated either by the kubelet returning from partitioned state, or deletion of the Node object, or deletion of the instance in the cloud provider, or force deletion of the pod from the api-server). This has the desirable outcome of "fencing" to prevent "split brain" scenarios.
* For all other existing controllers except StatefulSet , this has no effect on the ability of the controller to replace pods because the controllers do not reuse pod names (they use generate-name).
* User-written controllers that reuse names of pod objects should evaluate this change.
```

Automatic merge from submit-queue

AWS: recognize us-east-2 region

Automatic merge from submit-queue

CRI: Rename container/sandbox states

The enum constants are not namespaced. The shorter, unspecifc names are likely
to cause naming conflicts in the future.

Also replace "SandBox" with "Sandbox" in the API for consistency.

/cc @kubernetes/sig-node

Automatic merge from submit-queue

support Azure disk dynamic provisioning

azure disk dynamic provisioning

A screen shot 

``` console
$ kubectl create -f examples/experimental/persistent-volume-provisioning/azure-dd.yaml
storageclass "slow" created
$ kubectl create -f examples/experimental/persistent-volume-provisioning/claim1.json
persistentvolumeclaim "claim1" created
$ kubectl describe pvc
Name:       claim1
Namespace:  default
Status:     Bound
Volume:     pvc-de7150d1-6a37-11e6-aec9-000d3a12e034
Labels:     <none>
Capacity:   3Gi
Access Modes:   RWO
$ kubectl create -f pod.yaml
replicationcontroller "nfs-server" created
$ kubectl describe pod
Name:       nfs-server-b9w6x
Namespace:  default
Node:       rootfs-dev/172.24.0.4
Start Time: Wed, 24 Aug 2016 19:46:21 +0000
Labels:     role=nfs-server
Status:     Running
IP:     172.17.0.2
Controllers:    ReplicationController/nfs-server
Containers:
  nfs-server:
    Container ID:   docker://be6f8c0e26dc896d4c53ef0d21c9414982f0b39a10facd6b93a255f9e1c3806c
    Image:      nginx
    Image ID:       docker://bfdd4ced794ed276a28cf56b233ea58dec544e9ca329d796cf30b8bcf6d39b3f
    Port:       
    State:      Running
      Started:      Wed, 24 Aug 2016 19:49:19 +0000
    Ready:      True
    Restart Count:  0
    Volume Mounts:
      /exports from mypvc (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9o0fj (ro)
    Environment Variables:  <none>
Conditions:
  Type      Status
  Initialized   True 
  Ready     True 
  PodScheduled  True 
Volumes:
  mypvc:
    Type:   PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  claim1
    ReadOnly:   false
  default-token-9o0fj:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-9o0fj
QoS Class:  BestEffort
Tolerations:    <none>
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath           Type        Reason      Message
  --------- --------    -----   ----            -------------           --------    ------      -------
  11m       11m     1   {default-scheduler }                    Normal      Scheduled   Successfully assigned nfs-server-b9w6x to rootfs-dev
  9m        9m      1   {kubelet rootfs-dev}                    Warning     FailedMount Unable to mount volumes for pod "nfs-server-b9w6x_default(6eb7fd98-6a33-11e6-aec9-000d3a12e034)": timeout expired waiting for volumes to attach/mount for pod "nfs-server-b9w6x"/"default". list of unattached/unmounted volumes=[mypvc]
  9m        9m      1   {kubelet rootfs-dev}                    Warning     FailedSync  Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "nfs-server-b9w6x"/"default". list of unattached/unmounted volumes=[mypvc]
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Pulling     pulling image "nginx"
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Pulled      Successfully pulled image "nginx"
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Created     Created container with docker id be6f8c0e26dc
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Started     Started container with docker id be6f8c0e26dc

```

@colemickens @brendandburns

Automatic merge from submit-queue

Add FeatureGates field to KubeletConfiguration

This threads the `--feature-gates` flag through the `KubeletConfiguration` object and also allows setting feature gates via dynamic Kubelet configuration. 

/cc @jlowdermilk

Automatic merge from submit-queue

e2e.go/kops: Bump timeout to 20m, fix KUBERNETES_PROVIDER

**What this PR does / why we need it**: I don't have a ton of proof, but I think https://k8s-testgrid.appspot.com/google-aws#kops-aws-updown builds 4045-4047 are just AWS and DNS slowness.

In addition, my original PR was meant to change `KUBERNETES_PROVIDER` based on `KUBERNETES_CONFORMANCE_PROVIDER`.

Automatic merge from submit-queue

kubeadm preflight checks: Warn user if connections to API or Discovery are going to be over proxy

**What this PR does / why we need it**: Continuing discussion from PR #35044, new version will provide warning if kubeadm run in environment where http connections would go over proxy.
Most of the time, it is not expected behaviour and leads to situations like in #34695

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #34695

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```

kubeadm during initialization of master and slave nodes need to make
several API calls directly to the node where it is running or master.
In environments with http/https proxies, user might accidentally
have configuration where connections to API would go over proxy instead
of directly.

User can re-run kubeadm with corrected NO_PROXY variable. Example:

  $ NO_PROXY=* kubeadm join ...

Along the way: Fix ginkgo-e2e.sh. This change got dropped in the
original PR, but it was meant to allow a conformance-style
kubectl-auth test and still have a legit cloud provider.

Revert "fixed some issues with kubectl set resources"

Automatic merge from submit-queue

Disruption e2e: wait for running pods in the table test, too

**What this PR does / why we need it**: Makes the `DisruptionController` tests less flaky.
**Which issue this PR fixes**: fixes #34032

The enum constants are not namespaced. The shorter, unspecifc names are likely
to cause naming conflicts in the future.

Also replace "SandBox" with "Sandbox" in the API.

[Federation][(Un)join-00] Implement federation/cluster resource generator.

Automatic merge from submit-queue

Remove an unnecessary log line in kubelet

Automatic merge from submit-queue

e2e: Fix GetReadySchedulableNodesOrDie for taints

**What this PR does / why we need it**:

This changes framework.GetReadySchedulableNodesOrDie and
framework.GetMasterAndWorkerNodesOrDie so that nodes that can't take a
generic fake pod due to a taint/toleration mismatch aren't returned.

This is a rehash of #35210, but pulls in the scheduler code.

**Which issue this PR fixes**: c.f. #35210

**Special notes for your reviewer**: I think it's gross that we keep having to manually compute this in e2es. Maybe we need a bug for that?

Automatic merge from submit-queue

Migrate Network Logic into runtime II

- [x] Probing
- [x] Bootstrap CIDR
- [x] Setup/Teardown
- [ ] ~~Hostports~~
- [ ] ~~BWShaping~~

https://github.com/kubernetes/kubernetes/pull/34780

Automatic merge from submit-queue

Eviction manager evicts based on inode consumption

Fixes: #32526 Integrate Cadvisor per-container inode stats into the summary api.  Make the eviction manager act based on inode consumption to evict pods using the most inodes.

This PR is pending on a cadvisor godeps update which will be included in PR #35136

This changes framework.GetReadySchedulableNodesOrDie and
framework.GetMasterAndWorkerNodesOrDie so that nodes that can't take a
generic fake pod due to a taint/toleration mismatch aren't returned.

This is a rehash of #35210, but pulls in the scheduler code.

Automatic merge from submit-queue

Add tooling to generate listers

Add lister-gen tool to auto-generate listers. So far this PR only demonstrates replacing the manually-written `StoreToLimitRangeLister` with the generated `LimitRangeLister`, as it's a small and easy swap.

cc @deads2k @liggitt @sttts @nikhiljindal @lavalamp @smarterclayton @derekwaynecarr  @kubernetes/sig-api-machinery @kubernetes/rh-cluster-infra

Automatic merge from submit-queue

Only set sysctls for infra containers

We did set the sysctls for each container in a pod. This opens up a way to set un-whitelisted sysctls during upgrade from v1.3:
- set annotation in v1.3 with an un-whitelisted sysctl. Set restartPolicy=Always
- upgrade cluster to v1.4
- kill container process
- un-whitelisted sysctl is set on restart of the killed container.

kubeadm during initialization of master and slave nodes need to make
several API calls directly to the node where it is running or master.
In environments with http/https proxies, user might accidentally
have configuration where connections to API would go over proxy instead
of directly.

User can re-run kubeadm with corrected NO_PROXY variable. Example:

  $ NO_PROXY=* kubeadm join ...

Fix density test in large clusters

Automatic merge from submit-queue

kubeadm: lower default component logging level

v=4 is likely to causer perf issues and v=2 is the default in most other setups including scale e2e. ref https://github.com/kubernetes/kubernetes/issues/26637#issuecomment-256859739

Automatic merge from submit-queue

remove non-reuseable bits of MasterServer

Scrub `master.go` again.  I think I'm pretty happy with this shape.  I may promote `InstallAPIs` since we're likely to want it downstream.

Automatic merge from submit-queue

SELinux Overhaul

Overhauls handling of SELinux in Kubernetes.  TLDR: Kubelet dir no longer has to be labeled `svirt_sandbox_file_t`.

Fixes #33351 and #33510.  Implements #33951.

Automatic merge from submit-queue

Add SNI support to the apiserver

This PR adds the `--tls-sni-key-cert` flag to the apiserver. It can be passed multiple times in the following ways:

``` shell
$ apiserver \
    --tls-sni-cert-key '*.example.com,example.com: example.key,example.crt' \
    --tls-sni-cert-key 'foo.key,foo.crt'
```

The first variant explicitly sets the accepted domain names, the second variant reads the common names and DNS names from the certificate itself.

If no domain name matches, the existing certificate (`--tls-cert-file`) is used.

``` golang
    fs.Var(config.NewNamedCertKeyArray(&s.SNICertKeys), "tls-sni-cert-key", ""+
        "A pair of x509 certificate and private key file paths, optionally prefixed with a list of "+
        "domain patterns which are fully qualified domain names, possibly with prefixed wildcard "+
        "segments. If no domain patterns are provided, the names of the certificate are "+
        "extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns "+
        "trump over extracted names. For multiple key/certificate pairs, use the "+
        "--tls-sni-key-cert multiple times. "+
        "Examples: \"example.key,example.crt\" or \"*.foo.com,foo.com:foo.key,foo.crt\".")
```

``` release-note
Add SNI support to the apiserver

Pass multiple certificates and domain name patterns with `--tls-sni-cert-key` and the right certificate will be chosen depending on the url the client is using.
```

Automatic merge from submit-queue

Remove archived clientset

We have [client-go](https://github.com/kubernetes/client-go) now, no need to keep archived clientset in master of k8s.io/kubernetes.

Also updated the client-gen README.

Fix https://github.com/kubernetes/kubernetes/issues/35803.

cc @mikedanese 

```release-note
The main kubernetes repository stops hosting archived version of released clients. Please use [client-go](https://github.com/kubernetes/client-go).
```

Automatic merge from submit-queue

break endless loop in vagrant cluster

reference to [vagrant.md](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/local-cluster/vagrant.md) troubleshooting ,  I guess we could set a time to return from endless loop to register with cloud provider.