Also, add unit tests for `kubefed unjoin`.

Automatic merge from submit-queue

[Federation][join-01] Implement `kubefed join` command.

Supersedes PR #35155.

Please review only the last commit here. This is based on PR #35492 which will be reviewed independently.

I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.

Design Doc: PR #34484

cc @kubernetes/sig-cluster-federation @quinton-hoole @mwielgus

Automatic merge from submit-queue

Making the pod.alpha.kubernetes.io/initialized annotation optional in PetSet pods

**What this PR does / why we need it**: As of now, the absence of the annotation `pod.alpha.kubernetes.io/initialized` in PetSets causes the PetSet controller to effectively "pause". Being a debug hook, users expect that its absence has no effect on the working of a PetSet. This PR inverts the logic so that we let the PetSet controller operate as expected in the absence of the annotation.
Letting the annotation remain alpha seems ok. Renaming it to something more meaningful needs further discussion.

**Which issue this PR fixes** _(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)_: fixes https://github.com/kubernetes/kubernetes/issues/35498

**Special notes for your reviewer**: 

**Release note**:

``` release-note
The annotation "pod.alpha.kubernetes.io/initialized" on StatefulSets (formerly PetSets) is now optional and only encouraged for debug use.
```

cc @erictune @smarterclayton @bprashanth @kubernetes/sig-apps 
@kow3ns The examples will need to be cleaned up as well I think later on to remove them.

Automatic merge from submit-queue

Scheduler Reduce functions can use PriorityMetadata

Automatic merge from submit-queue

Extend etcd migration logs

Ref #20504

Automatic merge from submit-queue

Fix build break on non-Linux OS introduced in 87aaf4c0

**What this PR does / why we need it**: simple fix for build breakage on non-Linux OS.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 
**Special notes for your reviewer**:

```console
+++ [1102 11:16:58] linux/amd64: go build started
+++ [1102 11:20:11] linux/amd64: go build finished
+++ [1102 11:16:58] darwin/amd64: go build started
# k8s.io/kubernetes/pkg/kubelet/dockershim/cm
pkg/kubelet/dockershim/cm/container_manager_unsupported.go:33: undefined: fmt in fmt.Errorf
+++ [1102 11:16:58] windows/amd64: go build started
# k8s.io/kubernetes/pkg/kubelet/dockershim/cm
pkg/kubelet/dockershim/cm/container_manager_unsupported.go:33: undefined: fmt in fmt.Errorf
Makefile:79: recipe for target 'all' failed
make[1]: *** [all] Error 1
Makefile:255: recipe for target 'cross' failed
make: *** [cross] Error 1
Makefile:239: recipe for target 'release' failed
make: *** [release] Error 1
```

**Release note**:
```release-note
NONE
```

Revert "Make master components in kubemark monitored by supervisord"

Automatic merge from submit-queue

split scheduler priorities into separate files

In the current state it's really hard to find a thing one is looking for, if he doesn't know already know where to look. cc @davidopp

Automatic merge from submit-queue

Create multiple clients in Density test and remove unused 'interval'

Fix #36039

Automatic merge from submit-queue

Hide groups with new versions from old kubectl

Fix https://github.com/kubernetes/kubernetes/issues/35791

**What caused the bug?**

In 1.5, we are going to graduate Policy and Apps to beta. Old version kubectl doesn't has the new versions built-in, its TRP dynamic discover thinks Policy/v1beta1 is a TPR, and tried to register it in kubectl's scheme. The registration failed because Policy group already exist, because kubectl had registered Policy.v1alpha1.

**How does this PR fix the bug?**

This PR let the API server hides Policy and Apps from old version kubectl, so TPR discovery won't see them.

Old version kubectl doesn't know about Policy/v1beta1 or Apps/v1beta1, and v1alpha1 will be removed, so old version kubectl won't work for Policy or Apps anyway, so this PR does not cause any function loss.

@kubernetes/sig-api-machinery @liggitt @smarterclayton @deads2k @janetkuo @mwielgus

Automatic merge from submit-queue

Make master components in kubemark monitored by supervisord

Automatic merge from submit-queue

kubeadm: move defaulting to the api group

ref #35796

@pires @kubernetes/sig-cluster-lifecycle

Automatic merge from submit-queue

Support events in restore script

Ref #20504

Automatic merge from submit-queue

Log portal IP in error info.

Log portal IP in error info. This will help when debugging.

Automatic merge from submit-queue

Align with other cli descriptions

**What this PR does / why we need it**:
Align with other cli descriptions

**Special notes for your reviewer**:
![image](https://cloud.githubusercontent.com/assets/20062886/19887803/adbeee16-a065-11e6-9933-98bfff784086.png)
Signed-off-by: yupeng <yu.peng36@zte.com.cn>

Automatic merge from submit-queue

Fix typo: attachement -> attachment

Fix panics in density test.

Automatic merge from submit-queue

dockershim: move docker to the given cgroup

Automatic merge from submit-queue

Only set sysctls in infra container in dockershim

Dockershim counterpart of https://github.com/kubernetes/kubernetes/pull/32383.

Fix typos

Automatic merge from submit-queue

Update verify-openapi-spec script to check for extra generated spec

hack/verify-openapi-spec.sh only check for existing spec changes. If for some reason (here most probably I forgot to delete a file in api/openapi-spec folder in #35388 after a rebase) there is an old spec exists in the spec folder, it won't panic but it should. This resulted in an unused out of date v1.spec file in the api/openapi-spec folder that this PR also removes.

Automatic merge from submit-queue

CRI: Handle empty container name in dockershim.

Fixes https://github.com/kubernetes/kubernetes/issues/35924.

Dead container may have no name, we should handle this properly.

@yujuhong @bprashanth

Automatic merge from submit-queue

make ./pkg/client/listers compile

currently compilation is broken

```
$ go install ./pkg/client/listers/...
# k8s.io/kubernetes/pkg/client/listers/apps/v1alpha1
pkg/client/listers/apps/v1alpha1/zz_generated.statefulset.go:89: undefined: apps in apps.Resource
# k8s.io/kubernetes/pkg/client/listers/autoscaling/v1
pkg/client/listers/autoscaling/v1/zz_generated.horizontalpodautoscaler.go:89: undefined: autoscaling in autoscaling.Resource
# k8s.io/kubernetes/pkg/client/listers/batch/v2alpha1
pkg/client/listers/batch/v2alpha1/zz_generated.job.go:89: undefined: batch in batch.Resource
pkg/client/listers/batch/v2alpha1/zz_generated.scheduledjob.go:89: undefined: batch in batch.Resource
# k8s.io/kubernetes/pkg/client/listers/authentication/v1beta1
pkg/client/listers/authentication/v1beta1/zz_generated.tokenreview.go:63: undefined: authentication in authentication.Resource
# k8s.io/kubernetes/pkg/client/listers/batch/v1
pkg/client/listers/batch/v1/zz_generated.job.go:89: undefined: batch in batch.Resource
# k8s.io/kubernetes/pkg/client/listers/authorization/v1beta1
pkg/client/listers/authorization/v1beta1/zz_generated.localsubjectaccessreview.go:89: undefined: authorization in authorization.Resource
pkg/client/listers/authorization/v1beta1/zz_generated.selfsubjectaccessreview.go:63: undefined: authorization in authorization.Resource
pkg/client/listers/authorization/v1beta1/zz_generated.subjectaccessreview.go:63: undefined: authorization in authorization.Resource
# k8s.io/kubernetes/pkg/client/listers/certificates/v1alpha1
pkg/client/listers/certificates/v1alpha1/zz_generated.certificatesigningrequest.go:63: undefined: certificates in certificates.Resource
# k8s.io/kubernetes/pkg/client/listers/policy/v1alpha1
pkg/client/listers/policy/v1alpha1/zz_generated.poddisruptionbudget.go:89: undefined: policy in policy.Resource
# k8s.io/kubernetes/pkg/client/listers/core/v1
pkg/client/listers/core/v1/zz_generated.componentstatus.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.configmap.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.endpoints.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.event.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.limitrange.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.namespace.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.node.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.persistentvolume.go:62: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.persistentvolumeclaim.go:89: undefined: api in api.Resource
pkg/client/listers/core/v1/zz_generated.pod.go:89: undefined: api
pkg/client/listers/core/v1/zz_generated.pod.go:89: too many errors
# k8s.io/kubernetes/pkg/client/listers/imagepolicy/v1alpha1
pkg/client/listers/imagepolicy/v1alpha1/zz_generated.imagereview.go:63: undefined: imagepolicy in imagepolicy.Resource
# k8s.io/kubernetes/pkg/client/listers/rbac/v1alpha1
pkg/client/listers/rbac/v1alpha1/zz_generated.clusterrole.go:63: undefined: rbac in rbac.Resource
pkg/client/listers/rbac/v1alpha1/zz_generated.clusterrolebinding.go:63: undefined: rbac in rbac.Resource
pkg/client/listers/rbac/v1alpha1/zz_generated.role.go:89: undefined: rbac in rbac.Resource
pkg/client/listers/rbac/v1alpha1/zz_generated.rolebinding.go:89: undefined: rbac in rbac.Resource
# k8s.io/kubernetes/pkg/client/listers/storage/v1beta1
pkg/client/listers/storage/v1beta1/zz_generated.storageclass.go:63: undefined: storage in storage.Resource
# k8s.io/kubernetes/pkg/client/listers/extensions/v1beta1
pkg/client/listers/extensions/v1beta1/zz_generated.daemonset.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.deployment.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.ingress.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.job.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.podsecuritypolicy.go:63: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.replicaset.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.scale.go:89: undefined: extensions in extensions.Resource
pkg/client/listers/extensions/v1beta1/zz_generated.thirdpartyresource.go:63: undefined: extensions in extensions.Resource
```
cc @ncdc @caesarxuchao

Automatic merge from submit-queue

Disable gci-mounter in cri node e2e tests

gci-mounter is still being validated and there are known issues. Do not enable it
for cri tests for now.

Automatic merge from submit-queue

CRI: Add kuberuntime container logs

Based on https://github.com/kubernetes/kubernetes/pull/34858.

The first 2 commits are from #34858. And the last 2 commits are new.

This PR added kuberuntime container logs support and add unit test for it.

I've tested all the functions manually, and I'll send another PR to write a node e2e test for container log.

**_Notice: current implementation doesn't support log rotation**_, which means that:
- It will not retrieve logs in rotated log file.
- If log rotation happens when following the log:
  - If the rotation is using create mode, we'll still follow the old file.
  - If the rotation is using copytruncate, we'll be reading at the original position and get nothing.

To solve these issues, kubelet needs to rotate the log itself, or at least kubelet should be able to control the the behavior of log rotator. These are doable but out of the scope of 1.5 and will be addressed in future release.

@yujuhong @feiskyer @yifan-gu 
/cc @kubernetes/sig-node

Automatic merge from submit-queue

Node controller to not force delete pods

Fixes https://github.com/kubernetes/kubernetes/issues/35145

- [x] e2e tests to test Petset, RC, Job.
- [x] Remove and cover other locations where we force-delete pods within the NodeController.

**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
Node controller no longer force-deletes pods from the api-server.

* For StatefulSet (previously PetSet), this change means creation of replacement pods is blocked until old pods are definitely not running (indicated either by the kubelet returning from partitioned state, or deletion of the Node object, or deletion of the instance in the cloud provider, or force deletion of the pod from the api-server). This has the desirable outcome of "fencing" to prevent "split brain" scenarios.
* For all other existing controllers except StatefulSet , this has no effect on the ability of the controller to replace pods because the controllers do not reuse pod names (they use generate-name).
* User-written controllers that reuse names of pod objects should evaluate this change.
```

Automatic merge from submit-queue

AWS: recognize us-east-2 region

Automatic merge from submit-queue

CRI: Rename container/sandbox states

The enum constants are not namespaced. The shorter, unspecifc names are likely
to cause naming conflicts in the future.

Also replace "SandBox" with "Sandbox" in the API for consistency.

/cc @kubernetes/sig-node

gci-mounter is still being validated and there are known issues. Do not enable it
for cri tests for now.

Automatic merge from submit-queue

support Azure disk dynamic provisioning

azure disk dynamic provisioning

A screen shot 

``` console
$ kubectl create -f examples/experimental/persistent-volume-provisioning/azure-dd.yaml
storageclass "slow" created
$ kubectl create -f examples/experimental/persistent-volume-provisioning/claim1.json
persistentvolumeclaim "claim1" created
$ kubectl describe pvc
Name:       claim1
Namespace:  default
Status:     Bound
Volume:     pvc-de7150d1-6a37-11e6-aec9-000d3a12e034
Labels:     <none>
Capacity:   3Gi
Access Modes:   RWO
$ kubectl create -f pod.yaml
replicationcontroller "nfs-server" created
$ kubectl describe pod
Name:       nfs-server-b9w6x
Namespace:  default
Node:       rootfs-dev/172.24.0.4
Start Time: Wed, 24 Aug 2016 19:46:21 +0000
Labels:     role=nfs-server
Status:     Running
IP:     172.17.0.2
Controllers:    ReplicationController/nfs-server
Containers:
  nfs-server:
    Container ID:   docker://be6f8c0e26dc896d4c53ef0d21c9414982f0b39a10facd6b93a255f9e1c3806c
    Image:      nginx
    Image ID:       docker://bfdd4ced794ed276a28cf56b233ea58dec544e9ca329d796cf30b8bcf6d39b3f
    Port:       
    State:      Running
      Started:      Wed, 24 Aug 2016 19:49:19 +0000
    Ready:      True
    Restart Count:  0
    Volume Mounts:
      /exports from mypvc (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9o0fj (ro)
    Environment Variables:  <none>
Conditions:
  Type      Status
  Initialized   True 
  Ready     True 
  PodScheduled  True 
Volumes:
  mypvc:
    Type:   PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  claim1
    ReadOnly:   false
  default-token-9o0fj:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-9o0fj
QoS Class:  BestEffort
Tolerations:    <none>
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath           Type        Reason      Message
  --------- --------    -----   ----            -------------           --------    ------      -------
  11m       11m     1   {default-scheduler }                    Normal      Scheduled   Successfully assigned nfs-server-b9w6x to rootfs-dev
  9m        9m      1   {kubelet rootfs-dev}                    Warning     FailedMount Unable to mount volumes for pod "nfs-server-b9w6x_default(6eb7fd98-6a33-11e6-aec9-000d3a12e034)": timeout expired waiting for volumes to attach/mount for pod "nfs-server-b9w6x"/"default". list of unattached/unmounted volumes=[mypvc]
  9m        9m      1   {kubelet rootfs-dev}                    Warning     FailedSync  Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "nfs-server-b9w6x"/"default". list of unattached/unmounted volumes=[mypvc]
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Pulling     pulling image "nginx"
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Pulled      Successfully pulled image "nginx"
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Created     Created container with docker id be6f8c0e26dc
  8m        8m      1   {kubelet rootfs-dev}    spec.containers{nfs-server} Normal      Started     Started container with docker id be6f8c0e26dc

```

@colemickens @brendandburns