Automatic merge from submit-queue

Add clientset support for federation e2e tests.

Only the last commit here needs review.

Depends on #26952.

cc @colhom @kubernetes/sig-cluster-federation

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Reduced flakiness of HPA e2e tests.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Reduced flakiness of HPA e2e tests.

Automatic merge from submit-queue

Use exponential estimator for Heapster addon resizer

cc @fgrzadkowski @mwielgus @roberthbailey 
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Reduced flakiness of HPA e2e tests.

Automatic merge from submit-queue

e2e: run empty_dir test we've been missing

otherwise it's just dead code, and this seemed unintentional

Automatic merge from submit-queue

Rework PV controller to use util/goroutinemap


@kubernetes/sig-storage 

Automatic merge from submit-queue

Let kubelet log the DeletionTimestamp if it's not nil in update

This helps to debug if it's the kubelet to blame when a pod is not deleted. 

Example output:
```
SyncLoop (UPDATE, "api"): "redis-master_default(c6782276-2dd4-11e6-b874-64510650ab1c):DeletionTimestamp=2016-06-08T23:58:12Z"
```

ref #26290
cc @Random-Liu 

Automatic merge from submit-queue

Update reason_cache.go, Get method operate lru cache not threadsafe

The reason_cache wrapped lru cache , lru cache modies linked list even for a get, should use WLock for both read and write

Automatic merge from submit-queue

Fix docker api version in kubelet

There are two variables `dockerv110APIVersion` and `dockerV110APIVersion` with
the same purpose, but different values. Remove the incorrect one and fix usage
in the file.

/cc @dchen1107 @Random-Liu 

Automatic merge from submit-queue

processor listener: fix locking in pop()

Currently the lock in processorListener is used to guard pendingNotifications. But in pop, it also locks around on select chan. This will block the goroutine with lock acquired.

This PR changes the lock to guard the correct section only.

Automatic merge from submit-queue

pkg/kubectl: add resource printers for rbac api group

This PR adds the necessary kubectl printers for the rbac api group which we overlooked in previous PRs.

cc @erictune 

Automatic merge from submit-queue

ResourceQuota BestEffort scope aligned with Pod level QoS

This aligns quota with the changes in kubelet and CLI.

So if quota allows 10 `BestEffort` pods, it will now track properly with what the user sees with changes in 1.3.

```
apiVersion: v1
kind: ResourceQuota
metadata:
  name: best-effort
spec:
  hard:
    pods: "10"
  scopes:
  - BestEffort
```

/cc @vishh @kubernetes/rh-cluster-infra 

Automatic merge from submit-queue

godep bump: github.com/evanphx/json-patch

Fixes #26890

Automatic merge from submit-queue

AWS: cache instances during service reload to avoid rate limiting on restart

Fixes #25610 by reducing redundant calls to DescribeInstances()
 
```release-note
* The AWS cloudprovider will cache results from DescribeInstances() if the set of nodes hasn't changed
```

Also move int/stringSlicesEqual from servicecontroller.go to pkg/util/slice

Make each integration test use different prefix

Automatic merge from submit-queue

e2e: actually check error when we fail to GET the scheduled pod

and GET the pod before we try to gracefully delete it.
and add a debug log.

ref #26224

Automatic merge from submit-queue

GCE provider: Log full contents of long operations

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()Dump JSON of long running (>1m) GCE operations.

Automatic merge from submit-queue

Extract interface for master endpoints reconciler.

Make the master endpoints reconciler an interface so its implementation can be overridden, if
desired.

xref #20975 #26574 

cc @kubernetes/sig-api-machinery @lavalamp @smarterclayton @pmorie @DirectXMan12 @wojtek-t @kubernetes/rh-cluster-infra 

Automatic merge from submit-queue

Fix system container detection

```release-note
Fix system container detection in kubelet on systemd.

This fixed environments where CPU and Memory Accounting were not enabled on the unit 
that launched the kubelet or docker from reporting the root cgroup when 
monitoring usage stats for those components.
```

Fixes https://github.com/kubernetes/kubernetes/issues/25909

/cc @vishh @ncdc @eparis @wojtek-t - this reverts the revert https://github.com/kubernetes/kubernetes/issues/26478

This should merge when node e2e setups were updated per https://github.com/kubernetes/kubernetes/pull/26289

Automatic merge from submit-queue

fix recursive & non-recursive kubectl get of generic output format

This PR fixes the issues with `kubectl get` in https://github.com/kubernetes/kubernetes/issues/26466

Changes made:
- fix printing when using the generic output format in both non-recursive & recurvise settings to ensure that errors are being shown
- add tests to check printing generic output in a **non-recursive** setting with non-existent pods
- clean up the **recursive** `kubectl get` tests

/cc @janetkuo

Automatic merge from submit-queue

Considering all nodes for the scheduler cache to allow lookups

Fixes the actual issue that led me to create https://github.com/kubernetes/kubernetes/issues/22554

Currently the nodes in the cache provided to the predicates excludes the unschedulable nodes using field level filtering for the watch results. This results in the above issue as the `ServiceAffinity` predicate uses the cached node list to look up the node metadata for a peer pod (another pod belonging to the same service). Since this peer pod could be currently hosted on a node that is currently unschedulable, the lookup could potentially fail, resulting in the pod failing to be scheduled.

As part of the fix, we are now including all nodes in the watch results and excluding the unschedulable nodes using `NodeCondition`

@derekwaynecarr PTAL

Automatic merge from submit-queue

shorten e2e_node test, fix pod ready test issue

Fix #25703, #25812.
I fixed https://github.com/kubernetes/kubernetes/pull/26070 and open this PR.

@liangchenye 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Port the downward api test to the node e2e suite

Also extend the framework to allow a custom client config loading function, so
that the node e2e suite can reuse the same framework across tests.

This fixes #26609

/cc @timstclair @pwittrock 

Automatic merge from submit-queue

Added hpa/v1 generator to kubectl autoscale

ref #21577

```release-note
New default horizontalpodautoscaler/v1 generator for kubectl autoscale.
Use autoscaling/v1 in kubectl by default.
```


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Move /seccomp/ into domain prefix in seccomp annotations

Fixes #26610.

/cc @mdshuai @ncdc @jfrazelle

Automatic merge from submit-queue

Add e2e test for node problem detector

Based on https://github.com/kubernetes/node-problem-detector/pull/12.

This PR added e2e test for node problem detector. It starts a node problem detector with test configuration, and test its functionality.

Question:
* Should this be a node e2e test or e2e test? 
* Should we mark this as Serial? The test should not affect other tests, but it will add a `TestConditon` in node conditions and remove in cleanup.

@dchen1107
/cc @kubernetes/sig-node 
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Sets IgnoreUnknown=1 in CNI_ARGS

```release-note
release-note-none
```

K8 uses CNI_ARGS to pass pod namespace, name and infra container
id to the CNI network plugin. CNI logic will throw an error
if these args are not known to it, unless the user specifies
IgnoreUnknown as part of CNI_ARGS. This PR sets IgnoreUnknown=1
to prevent the CNI logic from erroring and blocking pod setup.
https://github.com/appc/cni/pull/158
https://github.com/appc/cni/issues/126

Automatic merge from submit-queue

In case of i/o timeout error in kubectl e2e, try talking to the server again after 2s before dying

Ref #26715

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

rkt: Error out when the gid is empty.

cc @kubernetes/rktnetes-maintainers 

Ref  https://github.com/appc/spec/issues/623

Automatic merge from submit-queue

Validation for illegal deployment strategy

Fixes #23677

@kubernetes/deployment 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

federation: Adding dnsprovider flags to federation-controller-manager

Automatic merge from submit-queue

federation service controller: fixing a bug so that existing services are created in newly registered clusters

A defect on federation service controller.
Steps to recreate:
1. boot federation control plane
2. create a service and then register a new cluster

Root cause:
the right sequence should be  
```
servicesToUpdate = s.serviceCache.allServices()
```
then
```
s.updateAllServicesToCluster(servicesToUpdate, newCluster) 
```
then
```
servicesToUpdate = s.updateDNSRecords(servicesToUpdate, newClusters)
```
Now the first two lines' sequence is on the contrary, so when updateDNSRecords return no error, the service will be removed from  servicesToUpdate, and updateAllServicesToCluster get nothing to update.

This PR make the call sequence correct.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Listing pods only once when getting pods for RS in deployment

Fixes #26834

1. Avoid ranging over RSes and then `List` pods of each RS. Instead, `List` pods of the deployment once, and then filter pods of each RS. 
2. Avoid using clientset to `List` pods in deployment controller. Use podStore instead. (TODO in some functions because the unit tests don't have podStore.)

@kubernetes/deployment 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Since appc requires gid to be non-empty today (https://github.com/appc/spec/issues/623),
we have to error out when gid is empty instead of using the root gid.

Revert "Revert "e2e-runner.sh: don't overwrite readonly var""