Automatic merge from submit-queue

Kubelet: implement GetPods for new runtime API

Implement GetPods for kuberuntime. Part of #28789 .

CC @yujuhong @Random-Liu

Automatic merge from submit-queue

e2e test for rescheduling critical pods

ref #29023

The test is disabled by default.

Automatic merge from submit-queue

AWS: More ELB attributes via service annotations

Replaces #25015 and addresses all of @justinsb's feedback therein. This is a new PR because I was unable to reopen #25015 to amend it.

I noticed recently that there is existing (but undocumented) precedent for the AWS cloud provider to manage ELB-specifc load balancer configuration based on service annotations.  In particular, one can _already_ designate an ELB as "internal" or enable PROXY protocol.

This PR extends this capability to the management of ELB attributes, which includes the following items:
* Access logs:
    * Enabled / disabled
    * Emit interval
    * S3 bucket name
    * S3 bucket prefix
* Connection draining:
    * Enabled / disabled
    * Timeout
* Connection:
    * Idle timeout
* Cross-zone load balancing:
    * Enabled / disabled

Some of these are possibly more useful than others.  Use cases that immediately come to mind:

* Enabling cross-zone load balancing is potentially useful for "Ubernetes Light," or anyone otherwise attempting to spread worker nodes around multiple AZs.
* Increasing idle timeout is useful for the benefit of anyone dealing with long-running requests. An example I personally care about would be git pushes to Deis' builder component.

Automatic merge from submit-queue

Remove the kubelet container as it's unmaintained

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

It removes the outdated and unmainained kubelet image, since we're using the hyperkube image instead.
Should be pretty straightforward to remove this.

@fgrzadkowski @thockin @mikedanese @pmorie

Automatic merge from submit-queue

[e2e density test] Fix unnecessary Delete RC requests when not running latency test

As the following code block
https://github.com/kubernetes/kubernetes/blob/master/test/e2e/density.go#L666-L670
shows, after running each density test case, it will attempt to delete "additional replication controllers" even though there is **no additional replication controller**.

When we are not running latency test, API Server will return "404 error code". So, I propose to move the above code block inside thedetermine statementsif `itArg.runLatencyTest{ }` , looks like:

```
if itArg.runLatencyTest {
  ...
  for i := 1; i <= nodeCount; i++ {
    name := additionalPodsPrefix + "-" + strconv.Itoa(i)
    c.ReplicationControllers(ns).Delete(name, nil)
  }
}
```
In this way, removing RC will be executed only if we set `itArg.runLatencyTest` to be `true`. It can avoid post some necessary requests to API Server.

Issuse is #30977

Automatic merge from submit-queue

Fix coding style

cc @pmorie

**What this PR does / why we need it**: Fixes case on a variable name, it's simple and adjust the code to the coding style.

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```NONE
```

Automatic merge from submit-queue

Fix description of AWS provisioner zones

Automatic merge from submit-queue

Impersonate user extra

Second commit builds on https://github.com/kubernetes/kubernetes/pull/30803.

This adds a restriction to `user.Info.Extra`, keys must be lower case.  This is because HTTP headers are case insensitive, so we can't be sure that we'll get the right case through proxies or even Go (the go library capitalizes after dashes).  I don't think anyone is using them, if they are, they'll need to update to properly plumb through an impersonation flow.

@kubernetes/sig-auth 
@ericchiang since you have background here.

Automatic merge from submit-queue

Updating version check for etcd

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

Currently, if you `make build`, etcd is not properly version checked because `etcd -version` does multi-line output. This output cannot be version compared. This small change fixes that.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

Automatic merge from submit-queue

Make DNS and Heapster critical pods

ref #29023
```release-note
DNS, Heapster and UI are critical addons
```

Automatic merge from submit-queue

Close websocket stream when client closes

Same problem as #30736, but for websocket streaming

Automatic merge from submit-queue

[GarbageCollector] Make Rate Limiter registration more efficient in GC

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:
Decrease the CPU consumption of the garbage collector

**Which issue this PR fixes** 
#30759

**Special notes for your reviewer**:
I observed dramatic improvement (dropped from 0.8cpu to 0.3cpu) in load test.

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```



@wojtek-t @lavalamp @gmarek

Automatic merge from submit-queue

	Don't validate selector that is already validated

Ref #30875

@timothysc

Automatic merge from submit-queue

Tweak resource consumption in density test

Fix: #30275

Automatic merge from submit-queue

Do not hold the lock for a long time

Followup to #30839.

I'm not convinced this is a super great idea but I'll throw it out and let others decide.

Ref https://github.com/kubernetes/minikube/issues/368
Ref #30759

Revert "Use netlink.SetPromiscOn instead of iproute2 command"

Automatic merge from submit-queue

AWS: Support HTTP->HTTP mode for ELB

**What this PR does / why we need it**:

Right now it is not possible to create an AWS ELB that listens for HTTP and where the backend pod also listens for HTTP.
I asked @justinsb in slack and he said that this seems to be an oversight, so I'd like to use this PR as a step towards solving this.

**Special notes for your reviewer**:

I've only added a simple unit test. Are any integration tests needed? I'm not familiar with the code base.

cc @therc

Automatic merge from submit-queue

Parameterize vpc name

Automatic merge from submit-queue

Fix caching of go dirs during build

Sometimes when you press `^C` during `make` the subsequent attempt to run `make` hangs due to zero-sized cache file for `ALL_GO_DIRS` var in Makefile.generated_files:
```
vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ KUBE_JUNIT_REPORT_DIR=/tmp/art KUBE_COVER=y make test
# hangs...
^CMakefile:279: recipe for target 'generated_files' failed
make: *** [generated_files] Interrupt

vagrant@devbox:~/work/kubernetes/src/k8s.io/kubernetes (master *%) $ ls -l .make/all_go_dirs.mk
-rw-rw-r-- 1 vagrant vagrant 0 Aug 18 15:03 .make/all_go_dirs.mk
```

 Corresponding process subtree looks like the following:
```
└─make test
    └─make -f Makefile.generated_files generated_files
        └─bash -c grep --color=never -l '+k8s:deepcopy-gen='  | xargs -n1 dirname | sort -u
            ├─grep --color=never -l +k8s:deepcopy-gen=
            ├─sort -u
            └─xargs -n1 dirname
```

Let's remove the cache file if `cache_go_dirs.sh` gets interrupted.

Automatic merge from submit-queue

Rename IsEmpty to Empty

addresses #21076

Automatic merge from submit-queue

e2e-runner: Get GCI builtin k8s version from GCS

**What this PR does / why we need it**:
The GCI release qualification tests use builtin k8s version to run tests. They used to get the builtin version by parsing descriptions of the test images, but that's unreliable because the description format could change. This PR makes it to read a file checked in GCS that has the exact k8s version number.

@adityakali @spxtr Can you review?

cc/ @kubernetes/goog-image

Automatic merge from submit-queue

Use netlink.SetPromiscOn instead of iproute2 command

Depends on https://github.com/vishvananda/netlink/pull/157
Related https://github.com/kubernetes/kubernetes/issues/26093

Automatic merge from submit-queue

Validate AppArmor annotations in the API server

This is important because it applies the same validation to the annotations that we would eventually want for the AppArmor fields, which will smooth the upgrade path.

/cc @pmorie @pweil- @jfrazelle @vishh

Automatic merge from submit-queue

[e2e test] Fix e2e test pause image hard code

Use `framework.GetPauseImageName(f.Client)` instead of hard code(such as `"gcr.io/google_containers/pause-amd64:3.0"`) to represent pause image name.

Related issus is #30967

Automatic merge from submit-queue

Add encryption to EBS dynamic provisioner

Resolves https://github.com/kubernetes/kubernetes/issues/30792

Adds encryption to the EBS cloud provider and provisioner.

Follow up to #29006 (all commits but the one in this PR will drop out).

@kubernetes/sig-storage 


```release-note
```

Automatic merge from submit-queue

kube-up: Allow IP restrictions for SSH and HTTPS API access on AWS.

<!--
Checklist for submitting a Pull Request

Please remove this comment block before submitting.

1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
   add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
   to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->

```release-note

```


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Closes #26661

Automatic merge from submit-queue

Add readyReplicas to replica sets

@bgrant0607 for the api changes

@bprashanth for the controllers changes

@deads2k fyi

Automatic merge from submit-queue

Node E2E: Remove fatal error in e2e_node_suite_test.go

Addresses https://github.com/kubernetes/kubernetes/issues/30779#issuecomment-240532190.

Currently we run node e2e test in parallel, and ginkgo makes sure that we only initialize test framework in the first test node.
However, because we throw out some fatal error during the initialization. Once there is an fatal error, the first test node will die immediately without reporting any error, and the other nodes will exit because the first node is gone with meaningless error.

If kubelet start fails, we'll get something like:
```
------------------------------
Failure [132.485 seconds]
[BeforeSuite] BeforeSuite 
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138

  BeforeSuite on Node 1 failed

  /usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
------------------------------
......
------------------------------
Failure [132.465 seconds]
[BeforeSuite] BeforeSuite 
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138

  BeforeSuite on Node 1 failed

  /usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
```

This PR replaces these fatal errors with gomega assertion, with this PR, we'll get:
```
Failure [132.482 seconds]
[BeforeSuite] BeforeSuite 
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138

  should be able to start node services.
  Expected success, but got an error:
      <*errors.errorString | 0xc8203351b0>: {
          s: "failed to run server start command \"/tmp/ginkgo869068712/e2e_node.test --run-services-mode --server-start-timeout 2m0s --report-dir  --node-name lantaol0.mtv.corp.google.com --disable-kubenet=true --cgroups-per-qos=false --manifest-path /tmp/node-e2e-pod221291440 --eviction-hard memory.available<250Mi\": exit status 255",
      }
      failed to run server start command "/tmp/ginkgo869068712/e2e_node.test --run-services-mode --server-start-timeout 2m0s --report-dir  --node-name lantaol0.mtv.corp.google.com --disable-kubenet=true --cgroups-per-qos=false --manifest-path /tmp/node-e2e-pod221291440 --eviction-hard memory.available<250Mi": exit status 255

  /usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:117
------------------------------
Failure [132.485 seconds]
[BeforeSuite] BeforeSuite 
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138

  BeforeSuite on Node 1 failed

  /usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
------------------------------
......
------------------------------
Failure [132.465 seconds]
[BeforeSuite] BeforeSuite 
/usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138

  BeforeSuite on Node 1 failed

  /usr/local/google/home/lantaol/workspace/src/k8s.io/kubernetes/test/e2e_node/e2e_node_suite_test.go:138
```

This is much more informative.

/cc @kubernetes/sig-node