Automatic merge from submit-queue

Expand documentation and TODOs in a few packages

I was reading through unfamiliar code and mostly added TODOs and expanded and clarified documentations.

There are a couple of things that are real code changes:
- Removed some unused constants
- Changed `workqueue.Parallize` to clamp the number of worker goroutines to the number of items to be processed.
- Added another unit test to `workqueue.queue`.  I thought I found a bug (I was wrong) and wrote a unit test to isolate.  I figure the extra test is worth keeping.

Automatic merge from submit-queue

CRI: general grammar/spelling/consistency cleanup

No semantic changes, but a lot of shuffling of docstrings to make things
more consistent. In particular, standardise on the zeroth-article (i.e.
prefer `// Version` to `// The version`) and ending all docstrings with
periods.


(This knowingly conflicts with #36446 and intentionally omits changing the
Annotations field - I'll rebase this or that respectively as necessary.)

Automatic merge from submit-queue

Ensure proper serialization of updates and creates in federation test watcher

Fix for finalizer test problems. The issue there was that the updates were coming out of order. It was caused by firing updates in new goroutines in test watcher. The proper solution is to order them in a queue and fire in order on a single goroutine.

Ref: https://github.com/kubernetes/kubernetes/issues/36473#issuecomment-259605870

cc: @nikhiljindal @madhusudancs

Automatic merge from submit-queue

Fix wrong comparison var in e2e_node density test

Automatic merge from submit-queue

Add support for SourceIP preservation in Azure LBs

@thockin @colemickens now that we are setting DSR for Azure LBs this should "just work"

Automatic merge from submit-queue

Fix getting cgroup pids

Fixes https://github.com/kubernetes/kubernetes/issues/35214, https://github.com/kubernetes/kubernetes/issues/33232

Verified manually, but I didn't have time to run all the e2e's yet (will check it in the morning).

This should be cherry-picked into 1.4, and merged into 1.5 (/cc @saad-ali )

```release-note
Fix fetching pids running in a cgroup, which caused problems with OOM score adjustments & setting the /system cgroup ("misc" in the summary API).
```

/cc @kubernetes/sig-node

Automatic merge from submit-queue

Wait for deployment in the storage in federated deployment controller test

cc: @nikhiljindal

Automatic merge from submit-queue

bump golang to 1.7.3

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: bumps the go version to 1.7.3

**Which issue this PR fixes** _(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)_: fixes #

**Special notes for your reviewer**:

**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

Automatic merge from submit-queue

Update https-nginx example

Update how to edit the configmap. Since we are not going to support `kubeectl replace configmap cmName --from-file=filename` in `v1.5`.
And update the `inotifywait` command.

Fixes: #36489

Automatic merge from submit-queue

Add dnsmasq-metrics to the standard DNS pod

Automatic merge from submit-queue

Do not handle AlreadyExists errors yet

Until we fix https://github.com/kubernetes/kubernetes/issues/29735 (use a new hashing algo) we should not handle AlreadyExists (was added recently in the perma-failed PR).

@kubernetes/deployment

Automatic merge from submit-queue

Update staging client-go

Executed commands:
```
git fetch --all
git merge upstream/master
godep restore
./hack/update-staging-client-go.sh
```
cc: @caesarxuchao @lavalamp

Automatic merge from submit-queue

Add e2e node test for log path

fixes #34661

A node e2e test to check if container logs files are properly created with right content.

Since the log files under `/var/log/containers` are actually symbolic of docker containers log files, we can not use a pod to mount them in and do check (symbolic doesn't supported by docker volume).

cc @Random-Liu

Automatic merge from submit-queue

SSL certificates for etcd cluster.

Added generation of SSL certificates for etcd cluster's internal communication.
Turned on on GCE (gci, trusty and debain).

Automatic merge from submit-queue

Fix resource constraints in density test

Fixes one source of flakes in #34911

Automatic merge from submit-queue

Fix Docker Registry image version to 2.5.1

`registry:2` is constantly being updated with new versions. This means there's a possibility that the image may be changed unintentionally. For example, when the Pod is rescheduled on nodes that does not already have the image, depending on the time of the pull, `registry:2` may result in different images.

Fix this to the latest `registry:2.5.1` instead to avoid this problem.

@uluyol @freehan

Added generation of SSL certificates for etcd cluster internal
communication. Turned on on gci & trusty.

Automatic merge from submit-queue

Increase master disk size in large clusters

Ref #34911

Automatic merge from submit-queue

Adding cascading deletion support to federation replicaset and deployments

Forked from https://github.com/kubernetes/kubernetes/pull/36330

Ref #33612
Adding cascading deletion support for federated replicasets and deployments.

```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated replicasets and deployments. Setting it to false while deleting a federated replicaset or deployment also deletes the corresponding resource from all registered clusters.
```

Automatic merge from submit-queue

Change master to advertise external IP in kubernetes service.

Change master to advertise external IP in kubernetes service.
In effect, in HA mode in case of multiple masters, IP of external load
balancer will be advertise in kubernetes service.

Automatic merge from submit-queue

Disable keep-alive for SSH tunnel health checks

We don't reuse the http client across health checks. Disabling the keep-alive to
properly close the connections.

This fixes #35207

Automatic merge from submit-queue

HPA: removed duplicated events, added events in all execution paths.

Automatic merge from submit-queue

kubectl: show node label if defined

We are moving towards marking master nodes as tainted, and not
necessarily unschedulable.  Further now we encourage users to taint
nodes, marking them unschedulable.

Thus the reliance on "Unschedulable" is not really a great indicator for
the master.

Instead, recognize the existing node 'role' markers, and surface them
where Unschedulable is (in the status).

We recognize:

 * a kubernetes.io/role label
 * a kubeadm.alpha.kubernetes.io/role label
~a taint with Key 'dedicated'~

Fix #33533

Automatic merge from submit-queue

Migrates addons from RCs to Deployments

Fixes #33698.

Below addons are being migrated:
- kube-dns
- GLBC default backend
- Dashboard UI
- Kibana

For the new deployments, the version suffixes are removed from their names. Version related labels are also removed because they are confusing and not needed any more with regard to how Deployment and the new Addon Manager works.

The `replica` field in `kube-dns` Deployment manifest is removed for the incoming DNS horizontal autoscaling feature #33239.

The `replica` field in `Dashboard` Deployment manifest is also removed because the rescheduler e2e test is manually scaling it.

Some resource limit related fields in `heapster-controller.yaml` are removed, as they will be set up by the `addon resizer` containers. Detailed reasons in #34513.

Three e2e tests are modified:
- `rescheduler.go`: Changed to resize Dashboard UI Deployment instead of ReplicationController.
- `addon_update.go`: Some namespace related changes in order to make it compatible with the new Addon Manager.
- `dns_autoscaling.go`: Changed to examine kube-dns Deployment instead of ReplicationController.

Both of above two tests passed on my own cluster. The upgrade process --- from old Addons with RCs to new Addons with Deployments --- was also tested and worked as expected.

The last commit upgrades Addon Manager to v6.0. It is still a work in process and currently waiting for #35220 to be finished. (The Addon Manager image in used comes from a non-official registry but it mostly works except some corner cases.)

@piosz @gmarek could you please review the heapster part and the rescheduler test?

@mikedanese @thockin 

cc @kubernetes/sig-cluster-lifecycle 

---

Notes:
- Kube-dns manifest still uses *-rc.yaml for the new Deployment. The stale file names are preserved here for receiving faster review. May send out PR to re-organize kube-dns's file names after this.
- Heapster Deployment's name remains in the old fashion(with `-v1.2.0` suffix) for avoiding describe this upgrade transition explicitly. In this way we don't need to attach fake apply labels to the old Deployments.

Automatic merge from submit-queue

Add traces to etcd3 code

This is to confirm that long PUT nodes we observe are really on the etcd "server+client" side and not somewhere in the middle in apiserver side.

Automatic merge from submit-queue

Node E2E: Reorganize node e2e output directories.

Fixes https://github.com/kubernetes/kubernetes/issues/35074.

This PR cleans up the result directory and workspace directory of node e2e test.

Local result directory:

```
/tmp/_artifacts/
        |----- build-log.txt  (build log)
        |----- *.xml  (junit xml file)
        |----- local/  (local run *.log)
        |----- hostname1/  (remote run *.log)
        |----- hostname2/
```

Workspace directory on test node:

```
/tmp/node-e2e-yyyy-mm-ddThh-mm-ss/
        |----- cluster/  (gci mounter)
        |----- cni/  (cni binary)
        |----- e2e_node.test  (test binary)
        |----- e2e_node_test.tar.gz  (test tar)
        |----- etcd060429031/  (etcd data directory)
        |----- ginkgo  (ginkgo binary)
        |----- kubelet (kubelet binary)
        |----- pod-manifest365096781/  (mirror pod directory)
        |----- results/  (test result directory)
```

@mtaufen 
/cc @kubernetes/sig-node

Automatic merge from submit-queue

Avoid setting S_ISGID on files in volumes

Some applications are having issues with setting the S_ISGID bit on files in volumes.  We intend to do this for directories so that the group ID is inherited, but not files for which S_ISGID indicates madatory file locking https://linux.die.net/man/2/stat

xref https://bugzilla.redhat.com/show_bug.cgi?id=1387306

@ncdc @derekwaynecarr @pmorie

Automatic merge from submit-queue

Adding cascading deletion support to more federation controllers

Ref #33612

Adding cascading deletion support for federated daemonsets and ingress.
The code is same as that for namespaces. Just ensuring that DeletionHelper functions are called at right places in these controllers.
e2e tests coming up in another PR.

cc @kubernetes/sig-cluster-federation @caesarxuchao @madhusudancs @mwielgus


```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated daemonsets and ingresses. Setting it to false while deleting a federated daemonset or ingress also deletes the corresponding resource from all registered clusters.
```

- Enables prometheus metrics on kube-dns
- Explicitly set v=0 logging for now

Automatic merge from submit-queue

Enable NFS and GlusterFS tests in both node and cluster e2e tests

This PR is to enable NFS and GlusterFS tests on both node and cluster
e2e tests.

It also change the code to use ExecCommandInPod instead of kubectl since
node does not have kubectl available