Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)

Select initializers from the dynamic configuration

Continues #36721

kubernetes/features#209

Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)

Generate protobuf for the audit API

Add protobuf generation & annotations for the advanced auditing API. We're not using them directly now, but settling on the field numbering will help with some 3rd party integrations.

For kubernetes/features#22

/cc @ihmccreery

Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)

Add group alias names to API resources to allow discovery

**What this PR does / why we need it**: 
Adds `GroupNames []string` to API resources, which represents the list of group aliases that every resource belongs to. 

Partially fixes https://github.com/kubernetes/kubernetes/issues/41353

This moves the logic of "all" (which currently [translates](https://github.com/kubernetes/kubernetes/blob/fbc94c089657045217999995aaf08e4025b53c52/pkg/kubectl/cmd/util/shortcut_restmapper.go#L106) to "pods,replicationcontrollers,services,...") to the server-side. Will allow clients like `kubectl` to discover group aliases instead of having it hardcoded and the API server to better handle consistency across multiple clients, version skew, etc; and will make "all" un-special and allow other groups to be created.

As a follow-up we'll patch `kubectl` to make groups aliases discoverable and the hardcoded list a fallback while we still have to support it.

Related to https://github.com/kubernetes/kubernetes/pull/42595#issuecomment-286839885.

**Release note**:
```release-note
Adds the `Categories []string` field to API resources, which represents the list of group aliases (e.g. "all") that every resource belongs to. 
```
@kubernetes/sig-api-machinery-misc @deads2k @bgrant0607

Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)

save kubelet bootstrap certs on GCI

Fix https://github.com/kubernetes/kubernetes/issues/46982

Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)

deflake CRD watch tests

Fixes https://github.com/kubernetes/kubernetes/issues/46446. Again...

This flake window is caused by the watch cache starting late.  This pull updates the test to do fancy list/create/watch/delete semantics to catch the problem.  In the field, this should be treated the same as any other "resourceVersion tool old" error and handled with a list/watch.  The test cannot be level driven, it is actually testing the edge behavior, so we have to do something weird like this.

@sttts @liggitt let's try this again...

Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)

Log more verbose when DisruptionController doesn't start.

Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)

Fix round-trip of Unstructured.OwnerReferences

**What this PR does / why we need it**:
Previously `setOwnerReference()` was storing pointers but `extractOwnerReference()` is expecting pointer fields as plain values so it cannot read those pointers. And hence you cannot read what you've just stored.

**Which issue this PR fixes**
#46817

**Special notes for your reviewer**:
This is similar to #43346.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)

While deleting a namespace, the TPR instances under this ns should be…

… deleted.

While deleting a namespace, the TPR instances under this ns should be deleted.

Fixed #46736 

**Release note**:
```
None
```

Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)

update copyed doc for advanced audit

doc for WithAudit is copyed from WithLegacyAudit, it's out of date.
This change update doc for these two functions.

**Release note**:

```
NONE
```

Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)

Added Bind method to Scheduler Extender

- only one extender can support the bind method
- if an extender supports bind, scheduler delegates the pod binding to the extender



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #41235

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)

apiserver: return BadRequest 400 for invalid query params

Fixes https://github.com/kubernetes/kubernetes/issues/39730
Replaces #40773.

Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)

Add KernelDeadlock condition to hollow NPD

Ref https://github.com/kubernetes/kubernetes/issues/44701

/cc @wojtek-t @gmarek

Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)

promote tls-bootstrap to beta

last commit of this PR.

Towards https://github.com/kubernetes/kubernetes/issues/46999

```release-note
Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag.
```

Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)

Multizone static pv test

**What this PR does / why we need it**:
Adds an e2e test for checking that pods get scheduled to the same zone as statically created PVs.  This tests the PersistentVolumeLabel admission controller, which adds zone and region labels when PVs are created.  As part of this, I also had to make changes to volume test utility code to pass in a zone parameter for creating PDs, and also had to add an argument to the e2e test program to accept a list of zones.

Fixes #46995

**Special notes for your reviewer**:
It's probably easier to review each commit separately.

**Release note**:

NONE

Automatic merge from submit-queue

Check live pod on cache miss before forbidding node deletion

Fixes #46971 flake

Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)

Add configuration for encryption providers

## Additions

Allows providing a configuration file (using flag `--experimental-encryption-provider-config`) to use the existing AEAD transformer (with multiple keys) by composing mutable transformer, prefix transformer (for parsing providerId), another prefix transformer (for parsing keyId), and AES-GCM transformers (one for each key). Multiple providers can be configured using the configuration file.

Example configuration:
```
kind: EncryptionConfig
apiVersion: v1
resources:
  - resources:
    - namespaces
    providers:
    - aes:
        keys:
        - name: key1
          secret: c2vjcmv0iglzihnly3vyzq==
        - name: key2
          secret: dghpcybpcybwyxnzd29yza==
    - identity: {}
```

Need for configuration discussed in:
#41939
[Encryption](https://github.com/destijl/community/blob/3418b4e4c6358f5dc747a37b90a97bc792f159ee/contributors/design-proposals/encryption.md)

**Pathway of a read/write request**:
1. MutableTransformer
2. PrefixTransformer reads the provider-id, and passes the request further if that matches.
3. PrefixTransformer reads the key-id, and passes the request further if that matches.
4. GCMTransformer tries decrypting and authenticating the cipher text in case of reads. Similarly for writes.

## Caveats
1. To keep the command line parameter parsing independent of the individual transformer's configuration, we need to convert the configuration to an `interface{}` and manually parse it in the transformer. Suggestions on better ways to do this are welcome.

2. Flags `--encryption-provider` and `--encrypt-resource` (both mentioned in [this document](https://github.com/destijl/community/blob/3418b4e4c6358f5dc747a37b90a97bc792f159ee/contributors/design-proposals/encryption.md) ) are not supported in this because they do not allow more than one provider, and the current format for the configuration file possibly supersedes their functionality.

3. Currently, it can be tested by adding `--experimental-encryption-provider-config=config.yml` to `hack/local-up-cluster.sh` on line 511, and placing the above configuration in `config.yml` in the root project directory.

Previous discussion on these changes:
https://github.com/sakshamsharma/kubernetes/pull/1

@jcbsmpsn @destijl @smarterclayton

## TODO
1. Investigate if we need to store keys on disk (per [encryption.md](https://github.com/destijl/community/blob/3418b4e4c6358f5dc747a37b90a97bc792f159ee/contributors/design-proposals/encryption.md#option-1-simple-list-of-keys-on-disk))
2. Look at [alpha flag conventions](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go)
3. Need to reserve `k8s:enc` prefix formally for encrypted data. Else find a better way to detect transformed data.

Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)

bazel: base kube-proxy image on debian-iptables instead of busybox + iptables

**What this PR does / why we need it**: the bazel-built kube-proxy image currently uses a custom base image made up of scratch + busybox + iptables + a few dependencies, while the official kube-proxy image is based off of the debian-iptables image.

This difference seems to cause some weird issues such as #46103, since the container layout doesn't look the same.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #46103, probably?

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/assign @mikedanese @spxtr @pipejakob 
/cc @Q-Lee @thockin @cblecker

Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)

Update cAdvisor version to v0.26.0

issue: #46658

I have requested a 1 day exception for code freeze.

/assign @dchen1107 

```release-note
Fix disk partition discovery for brtfs
Add ZFS support
Add overlay2 storage driver support
```

Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)

[GCE] Support internal load balancers

**What this PR does / why we need it**:
Allows users to expose K8s services externally of the K8s cluster but within their GCP network. 

Fixes #33483

**Important User Notes:**
- This is a beta feature. ILB could be enabled differently in the future. 
- Requires nodes having version 1.7.0+ (ILB requires health checking and a health check endpoint on kube-proxy has just been exposed)
- This cannot be used for intra-cluster communication. Do not call the load balancer IP from a K8s node/pod.  
- There is no reservation system for private IPs. You can specify a RFC 1918 address in `loadBalancerIP` field, but it could be lost to another VM or LB if service settings are modified.
- If you're running an ingress, your existing loadbalancer backend service must be using BalancingMode type `RATE` - not `UTILIZATION`. 
  - Option 1: With a 1.5.8+ or 1.6.4+ version master, delete all your ingresses, and re-create them.
  - Option 2: Migrate to a new cluster running 1.7.0. Considering ILB requires nodes with 1.7.0, this isn't a bad idea.
  - Option 3: Possible migration opportunity, but use at your own risk. More to come later.


**Reviewer Notes**:
Several files were renamed, so github thinks ~2k lines have changed. Review commits one-by-one to see the actual changes.

**Release note**:
```release-note
Support creation of GCP Internal Load Balancers from Service objects
```

Handle failure cases on startup gracefully to avoid causing cascading
errors and poor initialization in other components. Initial errors from
config load cause the initializer to pause and hold requests. Return
typed errors to better communicate failures to clients.

Add code to handle two specific cases - admin wants to bypass
initialization defaulting, and mirror pods (which want to bypass
initialization because the kubelet owns their lifecycle).

Automatic merge from submit-queue

HPA Status Conditions

This PR introduces conditions to the status of the HorizontalPodAutoscaler (in autoscaling/v2alpha1).  
The conditions whether or not the autoscaler is actively scaling, and why.  This gives greater visibility
into the *current* status of the autoscaler, similarly to how conditions work for pods, nodes, etc.

`kubectl describe` has been updated to the display the conditions affecting a given HPA.

Implements kubernetes/features#264 (alpha in 1.7)

**Release note**:
```release-note
Introduces status conditions to the HorizontalPodAutoscaler in autoscaling/v2alpha1, indicating the current status of a given HorizontalPodAutoscaler, and why it is or is not scaling.
```

make the health check wait for ready apiservices

- only one extender can support the bind method
- if an extender supports bind, scheduler delegates the pod binding to the extender

This commit updates the generated autoscaling files to be up-to-date
with the HPA status condition changes.

This commit updates `kubectl describe` to display the new HPA
status conditions.  This should make it easier for users to discern
the current state of the HPA.

This commit causes the HPA controller to set a variety of status
conditions using the new `Status.Conditions` field of
autoscaling/v2alpha1.  These provide insight into the current state
of the HPA, and generally correspond to similar events being emitted.

This commit adds the new API status conditions to the API types.
The field exists as a field in autoscaling/v2alpha1, and is
round-tripped through an annotation in autoscaling/v1.

Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)

Added node to persistent-volume-binder clusterrole

**What this PR does / why we need it**: Added missing permission to volume-binder clusterrole

**Which issue this PR fixes**: fixes #46770

**Special notes for your reviewer**: Non

**Release note**: Non

Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)

Add iptables lock-file mount to kube-proxy manifest

**What this PR does / why we need it**: kube-proxy is broken in make bazel-release. The new iptables binary uses a lockfile in "/run", but the directory doesn't exist. This causes iptables-restore to fail. We need to share the same lock-file amongst all containers, so mount the host /run dir.

This is similar to #46132 but expediency matters, since builds are broken.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #46103

**Special notes for your reviewer**:

**Release note**:

```release-note
```