Merge pull request #9458 from brendandburns/release-0.19

hack/test-go.sh

@@ -33,6 +33,7 @@ kube::test::find_dirs() {
           -o -wholename './release' \
           -o -wholename './target' \
           -o -wholename '*/Godeps/*' \
+          -o -wholename './release*' \
           -o -wholename '*/contrib/podex/*' \
           -o -wholename '*/test/e2e/*' \
           -o -wholename '*/test/integration/*' \

pkg/version/base.go

@@ -36,8 +36,8 @@ package version
 var (
 	// TODO: Deprecate gitMajor and gitMinor, use only gitVersion instead.
 	gitMajor     string = "0"              // major version, always numeric
-	gitMinor     string = "18.1+"          // minor version, numeric possibly followed by "+"
-	gitVersion   string = "v0.18.1-dev"    // version from git, output of $(git describe)
+	gitMinor     string = "19.0+"          // minor version, numeric possibly followed by "+"
+	gitVersion   string = "v0.19.0-dev"    // version from git, output of $(git describe)
 	gitCommit    string = ""               // sha1 from git, output of $(git rev-parse HEAD)
 	gitTreeState string = "not a git tree" // state of git tree, either "clean" or "dirty"
 )

release-0.19.0/docs/.files_generated

+kubectl.md
+kubectl_api-versions.md
+kubectl_cluster-info.md
+kubectl_config.md
+kubectl_config_set-cluster.md
+kubectl_config_set-context.md
+kubectl_config_set-credentials.md
+kubectl_config_set.md
+kubectl_config_unset.md
+kubectl_config_use-context.md
+kubectl_config_view.md
+kubectl_create.md
+kubectl_delete.md
+kubectl_describe.md
+kubectl_exec.md
+kubectl_expose.md
+kubectl_get.md
+kubectl_label.md
+kubectl_logs.md
+kubectl_namespace.md
+kubectl_port-forward.md
+kubectl_proxy.md
+kubectl_rolling-update.md
+kubectl_run.md
+kubectl_scale.md
+kubectl_stop.md
+kubectl_update.md
+kubectl_version.md

release-0.19.0/docs/README.md

+# Kubernetes Documentation
+
+**Note**
+This documentation is current for 0.19.0.
+
+Documentation for previous releases is available in their respective branches:
+   * [v0.18.1](https://github.com/GoogleCloudPlatform/kubernetes/tree/release-0.18/docs)
+   * [v0.17.1](https://github.com/GoogleCloudPlatform/kubernetes/tree/release-0.17/docs)
+
+* The [User's guide](user-guide.md) is for anyone who wants to run programs and services on an exisiting Kubernetes cluster.
+
+* The [Cluster Admin's guide](cluster-admin-guide.md) is for anyone setting up a Kubernetes cluster or administering it.
+
+* The [Developer guide](developer-guide.md) is for anyone wanting to write programs that access the kubernetes API,
+ write plugins or extensions, or modify the core code of kubernetes.
+
+* The [Kubectl Command Line Interface](kubectl.md) is a detailed reference on the `kubectl` CLI.
+
+* The [API object documentation](http://kubernetes.io/third_party/swagger-ui/) is a detailed description of all fields found in core API objects.
+
+* An overview of the [Design of Kubernetes](design)
+
+* There are example files and walkthroughs in the [examples](../examples) folder.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/README.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/README.md?pixel)]()

release-0.19.0/docs/accessing_the_api.md

+# Configuring APIserver ports
+
+This document describes what ports the kubernetes apiserver
+may serve on and how to reach them.  The audience is
+cluster administrators who want to customize their cluster
+or understand the details. 
+
+Most questions about accessing the cluster are covered
+in [Accessing the cluster](../docs/accessing-the-cluster.md).
+
+
+## Ports and IPs Served On
+The Kubernetes API is served by the Kubernetes APIServer process.  Typically,
+there is one of these running on a single kubernetes-master node.
+
+By default the Kubernetes APIserver serves HTTP on 2 ports:
+  1. Localhost Port
+    - serves HTTP
+    - default is port 8080, change with `-port` flag.
+    - defaults IP is localhost, change with `-address` flag.
+    - no authentication or authorization checks in HTTP
+    - protected by need to have host access
+  2. Secure Port
+    - default is port 443, change with `-secure_port`
+    - default IP is first non-localhost network interface, change with `-public_address_override`
+    - serves HTTPS.  Set cert with `-tls_cert_file` and key with `-tls_private_key_file`.
+    - uses token-file or client-certificate based [authentication](./authentication.md).
+    - uses policy-based [authorization](./authorization.md).
+  3. Removed: ReadOnly Port
+    - For security reasons, this had to be removed. Use the service account feature instead.
+
+## Proxies and Firewall rules
+
+Additionally, in some configurations there is a proxy (nginx) running
+on the same machine as the apiserver process.  The proxy serves HTTPS protected
+by Basic Auth on port 443, and proxies to the apiserver on localhost:8080. In
+these configurations the secure port is typically set to 6443.
+
+A firewall rule is typically configured to allow external HTTPS access to port 443.
+
+The above are defaults and reflect how Kubernetes is deployed to GCE using
+kube-up.sh.  Other cloud providers may vary.
+
+## Use Cases vs IP:Ports
+
+There are three differently configured serving ports because there are a
+variety of uses cases:
+   1. Clients outside of a Kubernetes cluster, such as human running `kubectl`
+      on desktop machine.  Currently, accesses the Localhost Port via a proxy (nginx)
+      running on the `kubernetes-master` machine.  Proxy uses bearer token authentication.
+   2. Processes running in Containers on Kubernetes that need to do read from
+      the apiserver.  Currently, these can use a service account.
+   3. Scheduler and Controller-manager processes, which need to do read-write
+      API operations.  Currently, these have to run on the operations on the
+      apiserver.  Currently, these have to run on the same host as the
+      apiserver and use the Localhost Port.  In the future, these will be
+      switched to using service accounts to avoid the need to be co-located.
+   4. Kubelets, which need to do read-write API operations and are necessarily
+      on different machines than the apiserver.  Kubelet uses the Secure Port
+      to get their pods, to find the services that a pod can see, and to
+      write events.  Credentials are distributed to kubelets at cluster
+      setup time.
+
+## Expected changes
+   - Policy will limit the actions kubelets can do via the authed port.
+   - Kubelets will change from token-based authentication to cert-based-auth.
+   - Scheduler and Controller-manager will use the Secure Port too.  They
+     will then be able to run on different machines than the apiserver.
+   - A general mechanism will be provided for [giving credentials to
+     pods](
+     https://github.com/GoogleCloudPlatform/kubernetes/issues/1907).
+   - Clients, like kubectl, will all support token-based auth, and the
+     Localhost will no longer be needed, and will not be the default.
+     However, the localhost port may continue to be an option for
+     installations that want to do their own auth proxy.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/accessing_the_api.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/accessing_the_api.md?pixel)]()

release-0.19.0/docs/admission_controllers.md

+# Admission Controllers
+
+## What are they?
+
+An admission control plug-in is a piece of code that intercepts requests to the Kubernetes
+API server prior to persistence of the object, but after the request is authenticated
+and authorized.  The plug-in code is in the API server process
+and must be compiled into the binary in order to be used at this time.
+
+Each admission control plug-in is run in sequence before a request is accepted into the cluster.  If
+any of the plug-ins in the sequence reject the request, the entire request is rejected immediately
+and an error is returned to the end-user.
+
+Admission control plug-ins may mutate the incoming object in some cases to apply system configured
+defaults.  In addition, admission control plug-ins may mutate related resources as part of request
+processing to do things like increment quota usage.
+
+## Why do I need them?
+
+Many advanced features in Kubernetes require an admission control plug-in to be enabled in order
+to properly support the feature.  As a result, a Kubernetes API server that is not properly
+configured with the right set of admission control plug-ins is an incomplete server and will not
+support all the features you expect.
+
+## How do I turn on an admission control plug-in?
+
+The Kubernetes API server supports a flag, ```admission_control``` that takes a comma-delimited,
+ordered list of admission control choices to invoke prior to modifying objects in the cluster.
+
+## What does each plug-in do?
+
+### AlwaysAdmit
+
+Use this plugin by itself to pass-through all requests.
+
+### AlwaysDeny
+
+Rejects all requests.  Used for testing.
+
+### DenyExecOnPrivileged
+
+This plug-in will intercept all requests to exec a command in a pod if that pod has a privileged container.
+
+If your cluster supports privileged containers, and you want to restrict the ability of end-users to exec
+commands in those containers, we strongly encourage enabling this plug-in.
+
+### ServiceAccount
+
+This plug-in implements automation for [serviceAccounts]( service_accounts.md).
+We strongly recommend using this plug-in if you intend to make use of Kubernetes ```ServiceAccount``` objects.
+
+### SecurityContextDeny
+
+This plug-in will deny any pod with a [SecurityContext](security_context.md) that defines options that were not available on the ```Container```.
+
+### ResourceQuota
+
+This plug-in will observe the incoming request and ensure that it does not violate any of the constraints
+enumerated in the ```ResourceQuota``` object in a ```Namespace```.  If you are using ```ResourceQuota```
+objects in your Kubernetes deployment, you MUST use this plug-in to enforce quota constraints.
+
+See the [resourceQuota design doc]( design/admission_control_resource_quota.md).
+
+It is strongly encouraged that this plug-in is configured last in the sequence of admission control plug-ins.  This is
+so that quota is not prematurely incremented only for the request to be rejected later in admission control.
+
+### LimitRanger
+
+This plug-in will observe the incoming request and ensure that it does not violate any of the constraints
+enumerated in the ```LimitRange``` object in a ```Namespace```.  If you are using ```LimitRange``` objects in
+your Kubernetes deployment, you MUST use this plug-in to enforce those constraints.
+
+See the [limitRange design doc]( design/admission_control_limit_range.md).
+
+### NamespaceExists
+
+This plug-in will observe all incoming requests that attempt to create a resource in a Kubernetes ```Namespace```
+and reject the request if the ```Namespace``` was not previously created.  We strongly recommend running
+this plug-in to ensure integrity of your data.
+
+### NamespaceAutoProvision (deprecated)
+
+This plug-in will observe all incoming requests that attempt to create a resource in a Kubernetes ```Namespace```
+and create a new ```Namespace``` if one did not already exist previously.
+
+We strongly recommend ```NamespaceExists``` over ```NamespaceAutoProvision```.
+
+### NamespaceLifecycle
+
+This plug-in enforces that a ```Namespace``` that is undergoing termination cannot have new content created in it.
+
+A ```Namespace``` deletion kicks off a sequence of operations that remove all content (pods, services, etc.) in that
+namespace.  In order to enforce integrity of that process, we strongly recommend running this plug-in.
+
+Once ```NamespaceAutoProvision``` is deprecated, we anticipate ```NamespaceLifecycle``` and ```NamespaceExists``` will
+be merged into a single plug-in that enforces the life-cycle of a ```Namespace``` in Kubernetes.
+
+## Is there a recommended set of plug-ins to use?
+
+Yes.
+
+For Kubernetes 1.0, we strongly recommend running the following set of admission control plug-ins (order matters):
+
+```shell
+--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/admission_controllers.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/admission_controllers.md?pixel)]()

release-0.19.0/docs/annotations.md

+# Annotations
+
+We have [labels](labels.md) for identifying metadata.
+
+It is also useful to be able to attach arbitrary non-identifying metadata, for retrieval by API clients such as tools, libraries, etc. This information may be large, may be structured or unstructured, may include characters not permitted by labels, etc. Such information would not be used for object selection and therefore doesn't belong in labels.
+
+Like labels, annotations are key-value maps.
+```
+"annotations": {
+  "key1" : "value1",
+  "key2" : "value2"
+}
+```
+
+Possible information that could be recorded in annotations:
+
+* fields managed by a declarative configuration layer, to distinguish them from client- and/or server-set default values and other auto-generated fields, fields set by auto-sizing/auto-scaling systems, etc., in order to facilitate merging
+* build/release/image information (timestamps, release ids, git branch, PR numbers, image hashes, registry address, etc.)
+* pointers to logging/monitoring/analytics/audit repos
+* client library/tool information (e.g. for debugging purposes -- name, version, build info)
+* other user and/or tool/system provenance info, such as URLs of related objects from other ecosystem components
+* lightweight rollout tool metadata (config and/or checkpoints)
+* phone/pager number(s) of person(s) responsible, or directory entry where that info could be found, such as a team website
+
+Yes, this information could be stored in an external database or directory, but that would make it much harder to produce shared client libraries and tools for deployment, management, introspection, etc.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/annotations.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/annotations.md?pixel)]()

release-0.19.0/docs/api.md

+# The Kubernetes API
+
+Primary system and API concepts are documented in the [User guide](user-guide.md).
+
+Overall API conventions are described in the [API conventions doc](api-conventions.md).
+
+Complete API details are documented via [Swagger](http://swagger.io/). The Kubernetes apiserver (aka "master") exports an API that can be used to retrieve the [Swagger spec](https://github.com/swagger-api/swagger-spec/tree/master/schemas/v1.2) for the Kubernetes API, by default at `/swaggerapi`, and a UI you can use to browse the API documentation at `/swagger-ui`. We also periodically update a [statically generated UI](http://kubernetes.io/third_party/swagger-ui/).
+
+Remote access to the API is discussed in the [access doc](accessing_the_api.md).
+
+The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. The [Kubectl](kubectl.md) command-line tool can be used to create, update, delete, and get API objects.
+
+Kubernetes also stores its serialized state (currently in [etcd](https://coreos.com/docs/distributed-configuration/getting-started-with-etcd/)) in terms of the API resources.
+
+Kubernetes itself is decomposed into multiple components, which interact through its API.
+
+## API changes
+
+In our experience, any system that is successful needs to grow and change as new use cases emerge or existing ones change. Therefore, we expect the Kubernetes API to continuously change and grow. However, we intend to not break compatibility with existing clients, for an extended period of time. In general, new API resources and new resource fields can be expected to be added frequently. Elimination of resources or fields will require following a deprecation process. The precise deprecation policy for eliminating features is TBD, but once we reach our 1.0 milestone, there will be a specific policy.
+
+What constitutes a compatible change and how to change the API are detailed by the [API change document](devel/api_changes.md).
+
+## API versioning
+
+Fine-grain resource evolution alone makes it difficult to eliminate fields or restructure resource representations. Therefore, Kubernetes supports multiple API versions, each at a different API path prefix, such as `/api/v1beta3`. These are simply different interfaces to read and/or modify the same underlying resources. In general, all API resources are accessible via all API versions, though there may be some cases in the future where that is not true. 
+
+Distinct API versions present more clear, consistent views of system resources and behavior than intermingled, independently evolved resources. They also provide a more straightforward mechanism for controlling access to end-of-lifed and/or experimental APIs.
+
+The [API and release versioning proposal](versioning.md) describes the current thinking on the API version evolution process.
+
+## v1beta1, v1beta2, and v1beta3 are deprecated; please move to v1 ASAP
+
+As of June 4, 2015, the Kubernetes v1 API has been enabled by default. The v1beta1 and v1beta2 APIs were deleted on June 1, 2015. v1beta3 is planned to be deleted on July 6, 2015.
+
+### v1 conversion tips (from v1beta3)
+
+We're working to convert all documentation and examples to v1. A simple [API conversion tool](cluster_management.md#switching-your-config-files-to-a-new-api-version) has been written to simplify the translation process. Use `kubectl create --validate` in order to validate your json or yaml against our Swagger spec.
+
+Changes to services are the most significant difference between v1beta3 and v1.
+* The `service.spec.portalIP` property is renamed to `service.spec.clusterIP`.
+* The `service.spec.createExternalLoadBalancer` property is removed. Specify `service.spec.type: "LoadBalancer"` to create an external load balancer instead.
+* The `service.spec.publicIPs` property is deprecated and now called `service.spec.deprecatedPublicIPs`. This property will be removed entirely when v1beta3 is removed. The vast majority of users of this field were using it to expose services on ports on the node. Those users should specify `service.spec.type: "NodePort"` instead. Read [External Services](services.md#external-services) for more info. If this is not sufficient for your use case, please file an issue or contact @thockin.
+
+Some other difference between v1beta3 and v1:
+* The `pod.spec.containers[*].privileged` and `pod.spec.containers[*].capabilities` properties are now nested under the `pod.spec.containers[*].securityContext` property. See [Security Contexts](security_context.md).
+* The `pod.spec.host` property is renamed to `pod.spec.nodeName`.
+* The `endpoints.subsets[*].addresses.IP` property is renamed to `endpoints.subsets[*].addresses.ip`.
+* The `pod.status.containerStatuses[*].state.termination` and `pod.status.containerStatuses[*].lastState.termination` properties are renamed to `pod.status.containerStatuses[*].state.terminated` and `pod.status.containerStatuses[*].state.terminated` respectively.
+* The `pod.status.Condition` property is renamed to `pod.status.conditions`.
+* The `status.details.id` property is renamed to `status.details.name`.
+
+### v1beta3 conversion tips (from v1beta1/2)
+
+Some important differences between v1beta1/2 and v1beta3:
+* The resource `id` is now called `name`.
+* `name`, `labels`, `annotations`, and other metadata are now nested in a map called `metadata`
+* `desiredState` is now called `spec`, and `currentState` is now called `status`
+* `/minions` has been moved to `/nodes`, and the resource has kind `Node`
+* The namespace is required (for all namespaced resources) and has moved from a URL parameter to the path: `/api/v1beta3/namespaces/{namespace}/{resource_collection}/{resource_name}`. If you were not using a namespace before, use `default` here.
+* The names of all resource collections are now lower cased - instead of `replicationControllers`, use `replicationcontrollers`.
+* To watch for changes to a resource, open an HTTP or Websocket connection to the collection query and provide the `?watch=true` query parameter along with the desired `resourceVersion` parameter to watch from.
+* The `labels` query parameter has been renamed to `label-selector`.
+* The container `entrypoint` has been renamed to `command`, and `command` has been renamed to `args`.
+* Container, volume, and node resources are expressed as nested maps (e.g., `resources{cpu:1}`) rather than as individual fields, and resource values support [scaling suffixes](resources.md#resource-quantities) rather than fixed scales (e.g., milli-cores).
+* Restart policy is represented simply as a string (e.g., `"Always"`) rather than as a nested map (`always{}`).
+* Pull policies changed from `PullAlways`, `PullNever`, and `PullIfNotPresent` to `Always`, `Never`, and `IfNotPresent`.
+* The volume `source` is inlined into `volume` rather than nested.
+* Host volumes have been changed from `hostDir` to `hostPath` to better reflect that they can be files or directories.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/api.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/api.md?pixel)]()

release-0.19.0/docs/authentication.md

+# Authentication Plugins
+
+Kubernetes uses client certificates, tokens, or http basic auth to authenticate users for API calls.
+
+Client certificate authentication is enabled by passing the `--client_ca_file=SOMEFILE`
+option to apiserver. The referenced file must contain one or more certificates authorities
+to use to validate client certificates presented to the apiserver. If a client certificate
+is presented and verified, the common name of the subject is used as the user name for the
+request.
+
+Token authentication is enabled by passing the `--token_auth_file=SOMEFILE` option
+to apiserver.  Currently, tokens last indefinitely, and the token list cannot
+be changed without restarting apiserver.  We plan in the future for tokens to
+be short-lived, and to be generated as needed rather than stored in a file.
+
+The token file format is implemented in `plugin/pkg/auth/authenticator/token/tokenfile/...`
+and is a csv file with 3 columns: token, user name, user uid.
+
+When using token authentication from an http client the apiserver expects an `Authorization`
+header with a value of `Bearer SOMETOKEN`.
+
+Basic authentication is enabled by passing the `--basic_auth_file=SOMEFILE`
+option to apiserver. Currently, the basic auth credentials last indefinitely,
+and the password cannot be changed without restarting apiserver. Note that basic
+authentication is currently supported for convenience while we finish making the
+more secure modes described above easier to use.
+
+The basic auth file format is implemented in `plugin/pkg/auth/authenticator/password/passwordfile/...`
+and is a csv file with 3 columns: password, user name, user id.
+
+When using basic authentication from an http client the apiserver expects an `Authorization` header
+with a value of `Basic BASE64ENCODEDUSER:PASSWORD`.
+
+## Plugin Development
+
+We plan for the Kubernetes API server to issue tokens
+after the user has been (re)authenticated by a *bedrock* authentication
+provider external to Kubernetes.  We plan to make it easy to develop modules
+that interface between kubernetes and a bedrock authentication provider (e.g.
+github.com, google.com, enterprise directory, kerberos, etc.)
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/authentication.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/authentication.md?pixel)]()

release-0.19.0/docs/authorization.md

+# Authorization Plugins
+
+
+In Kubernetes, authorization happens as a separate step from authentication.
+See the [authentication documentation](./authentication.md) for an 
+overview of authentication.
+
+Authorization applies to all HTTP accesses on the main apiserver port. (The
+readonly port is not currently subject to authorization, but is planned to be
+removed soon.)
+
+The authorization check for any request compares attributes of the context of
+the request, (such as user, resource, and namespace) with access
+policies.  An API call must be allowed by some policy in order to proceed.
+
+The following implementations are available, and are selected by flag:
+  - `--authorization_mode=AlwaysDeny`
+  - `--authorization_mode=AlwaysAllow`
+  - `--authorization_mode=ABAC`
+
+`AlwaysDeny` blocks all requests (used in tests).
+`AlwaysAllow` allows all requests; use if you don't need authorization.
+`ABAC` allows for user-configured authorization policy.  ABAC stands for Attribute-Based Access Control.
+
+## ABAC Mode
+### Request Attributes
+
+A request has 4 attributes that can be considered for authorization:
+  - user (the user-string which a user was authenticated as).
+  - whether the request is readonly (GETs are readonly)
+  - what resource is being accessed 
+    - applies only to the API endpoints, such as 
+        `/api/v1/namespaces/default/pods`.  For miscellaneous endpoints, like `/version`, the
+        resource is the empty string.
+  - the namespace of the object being access, or the empty string if the
+        endpoint does not support namespaced objects.
+
+We anticipate adding more attributes to allow finer grained access control and
+to assist in policy management.
+
+### Policy File Format
+
+For mode `ABAC`, also specify `--authorization_policy_file=SOME_FILENAME`.
+
+The file format is [one JSON object per line](http://jsonlines.org/).  There should be no enclosing list or map, just
+one map per line.
+
+Each line is a "policy object".  A policy object is a map with the following properties:
+  - `user`, type string; the user-string from `--token_auth_file`
+  - `readonly`, type boolean, when true, means that the policy only applies to GET
+      operations.
+  - `resource`, type string; a resource from an URL, such as `pods`.
+  - `namespace`, type string; a namespace string.
+
+An unset property is the same as a property set to the zero value for its type (e.g. empty string, 0, false).
+However, unset should be preferred for readability.
+
+In the future, policies may be expressed in a JSON format, and managed via a REST
+interface.
+
+### Authorization Algorithm
+
+A request has attributes which correspond to the properties of a policy object.
+
+When a request is received, the attributes are determined.  Unknown attributes
+are set to the zero value of its type (e.g. empty string, 0, false). 
+
+An unset property will match any value of the corresponding
+attribute.  An unset attribute will match any value of the corresponding property.
+
+The tuple of attributes is checked for a match against every policy in the policy file.
+If at least one line matches the request attributes, then the request is authorized (but may fail later validation).
+
+To permit any user to do something, write a policy with the user property unset.
+To permit an action Policy with an unset namespace applies regardless of namespace.
+
+### Examples
+ 1. Alice can do anything: `{"user":"alice"}`
+ 2. Kubelet can read any pods: `{"user":"kubelet", "resource": "pods", "readonly": true}`
+ 3. Kubelet can read and write events: `{"user":"kubelet", "resource": "events"}`
+ 4. Bob can just read pods in namespace "projectCaribou": `{"user":"bob", "resource": "pods", "readonly": true, "ns": "projectCaribou"}`
+
+[Complete file example](../pkg/auth/authorizer/abac/example_policy_file.jsonl)
+
+## Plugin Development
+
+Other implementations can be developed fairly easily.
+The APIserver calls the Authorizer interface:
+```go
+type Authorizer interface {
+  Authorize(a Attributes) error
+}
+```
+to determine whether or not to allow each API action.
+
+An authorization plugin is a module that implements this interface.
+Authorization plugin code goes in `pkg/auth/authorization/$MODULENAME`.
+
+An authorization module can be completely implemented in go, or can call out
+to a remote authorization service.  Authorization modules can implement
+their own caching to reduce the cost of repeated authorization calls with the
+same or similar arguments.  Developers should then consider the interaction between
+caching and revocation of permissions.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/authorization.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/authorization.md?pixel)]()

release-0.19.0/docs/availability.md

+# Availability
+
+This document collects advice on reasoning about and provisioning for high-availability when using Kubernetes clusters.
+
+## Failure modes
+
+This is an incomplete list of things that could go wrong, and how to deal with them.
+
+Root causes:
+  - VM(s) shutdown
+  - network partition within cluster, or between cluster and users.
+  - crashes in Kubernetes software 
+  - data loss or unavailability of persistent storage (e.g. GCE PD or AWS EBS volume).
+  - operator error misconfigures kubernetes software or application software.
+
+Specific scenarios:
+  - Apiserver VM shutdown or apiserver crashing
+    - Results
+      - unable to stop, update, or start new pods, services, replication controller
+      - existing pods and services should continue to work normally, unless they depend on the Kubernetes API
+  - Apiserver backing storage lost
+    - Results
+      - apiserver should fail to come up.
+      - kubelets will not be able to reach it but will continute to run the same pods and provide the same service proxying.
+      - manual recovery or recreation of apiserver state necessary before apiserver is restarted.
+  - Supporting services (node controller, replication controller manager, scheduler, etc) VM shutdown or crashes
+    - currently those are colocated with the apiserver, and their unavailability has similar consequences as apiserver
+    - in future, these will be replicated as well and may not be co-located
+    - they do not have own persistent state
+  - Node (thing that runs kubelet and kube-proxy and pods) shutdown
+    - Results
+      - pods on that Node stop running
+  - Kubelet software fault
+    - Results
+      - crashing kubelet cannot start new pods on the node
+      - kubelet might delete the pods or not
+      - node marked unhealthy
+      - replication controllers start new pods elsewhere
+  - Cluster operator error
+    - Results:
+      - loss of pods, services, etc
+      - lost of apiserver backing store
+      - users unable to read API
+      - etc
+
+Mitigations:
+- Action: Use IaaS providers automatic VM restarting feature for IaaS VMs.
+  - Mitigates: Apiserver VM shutdown or apiserver crashing
+  - Mitigates: Supporting services VM shutdown or crashes
+
+- Action use IaaS providers reliable storage (e.g GCE PD or AWS EBS volume) for VMs with apiserver+etcd.
+  - Mitigates: Apiserver backing storage lost
+
+- Action: Use Replicated APIserver feature (when complete: feature is planned but not implemented)
+  - Mitigates: Apiserver VM shutdown or apiserver crashing
+    - Will tolerate one or more simultaneous apiserver failures.
+  - Mitigates: Apiserver backing storage lost
+    - Each apiserver has independent storage.  Etcd will recover from loss of one member.  Risk of total data loss greatly reduced.
+
+- Action: Snapshot apiserver PDs/EBS-volumes periodically
+  - Mitigates: Apiserver backing storage lost
+  - Mitigates: Some cases of operator error
+  - Mitigates: Some cases of kubernetes software fault
+
+- Action: use replication controller and services in front of pods
+  - Mitigates: Node shutdown
+  - Mitigates: Kubelet software fault
+
+- Action: applications (containers) designed to tolerate unexpected restarts
+  - Mitigates: Node shutdown
+  - Mitigates: Kubelet software fault
+
+- Action: Multiple independent clusters (and avoid making risky changes to all clusters at once)
+  - Mitigates: Everything listed above.
+
+## Choosing Multiple Kubernetes Clusters
+
+You may want to set up multiple kubernetes clusters, both to
+have clusters in different regions to be nearer to your users; and to tolerate failures and/or invasive maintenance.
+
+### Scope of a single cluster
+
+On IaaS providers such as Google Compute Engine or Amazon Web Services, a VM exists in a
+[zone](https://cloud.google.com/compute/docs/zones) or [availability
+zone](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html).
+We suggest that all the VMs in a Kubernetes cluster should be in the same availability zone, because:
+  - compared to having a single global Kubernetes cluster, there are fewer single-points of failure
+  - compared to a cluster that spans availability zones, it is easier to reason about the availability properties of a
+    single-zone cluster.
+  - when the Kubernetes developers are designing the system (e.g. making assumptions about latency, bandwidth, or
+    correlated failures) they are assuming all the machines are in a single data center, or otherwise closely connected.
+
+It is okay to have multiple clusters per availability zone, though on balance we think fewer is better.
+Reasons to prefer fewer clusters are:
+  - improved bin packing of Pods in some cases with more nodes in one cluster.
+  - reduced operational overhead (though the advantage is diminished as ops tooling and processes matures).
+  - reduced costs for per-cluster fixed resource costs, e.g. apiserver VMs (but small as a percentage
+    of overall cluster cost for medium to large clusters).
+
+Reasons to have multiple clusters include:
+  - strict security policies requiring isolation of one class of work from another (but, see Partitioning Clusters
+    below).
+  - test clusters to canary new Kubernetes releases or other cluster software.
+
+### Selecting the right number of clusters
+The selection of the number of kubernetes clusters may be a relatively static choice, only revisted occasionally.
+By contrast, the number of nodes in a cluster and the number of pods in a service may be change frequently according to
+load and growth.
+
+To pick the number of clusters, first, decide which regions you need to be in to have adequete latency to all your end users, for services that will run
+on Kubernetes (if you use a Content Distribution Network, the latency requirements for the CDN-hosted content need not
+be considered).  Legal issues might influence this as well. For example, a company with a global customer base might decide to have clusters in US, EU, AP, and SA regions. 
+Call the number of regions to be in `R`.
+
+Second, decide how many clusters should be able to be unavailable at the same time, while still being available.  Call
+the number that can be unavailable `U`.  If you are not sure, then 1 is a fine choice.
+
+If it is allowable for load-balancing to direct traffic to any region in the event of a cluster failure, then 
+then you need `R + U` clusters.  If it is not (e.g you want to ensure low latency for all users in the event of a
+cluster failure), then you need to have `R * U` clusters (`U` in each of `R` regions).  In any case, try to put each cluster in a different zone.
+
+Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then
+you may need even more clusters.  Our [roadmap](http://docs.k8s.io/roadmap.md)
+calls for maximum 100 node clusters at v1.0 and maximum 1000 node clusters in the middle of 2015.
+
+## Working with multiple clusters
+
+When you have multiple clusters, you would typically create services with the same config in each cluster and put each of those
+service instances behind a load balancer (AWS Elastic Load Balancer, GCE Forwarding Rule or HTTP Load Balancer), so that
+failures of a single cluster are not visible to end users.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/availability.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/availability.md?pixel)]()

release-0.19.0/docs/cli-roadmap.md

+# Kubernetes CLI/Configuration Roadmap
+
+See also issues with the following labels:
+* [area/config-deployment](https://github.com/GoogleCloudPlatform/kubernetes/labels/area%2Fconfig-deployment)
+* [component/CLI](https://github.com/GoogleCloudPlatform/kubernetes/labels/component%2FCLI)
+* [component/client](https://github.com/GoogleCloudPlatform/kubernetes/labels/component%2Fclient)
+
+1. Create services before other objects, or at least before objects that depend upon them. Namespace-relative DNS mitigates this some, but most users are still using service environment variables. [#1768](https://github.com/GoogleCloudPlatform/kubernetes/issues/1768)
+1. Finish rolling update [#1353](https://github.com/GoogleCloudPlatform/kubernetes/issues/1353)
+  1. Friendly to auto-scaling [#2863](https://github.com/GoogleCloudPlatform/kubernetes/pull/2863#issuecomment-69701562)
+  1. Rollback (make rolling-update reversible, and complete an in-progress rolling update by taking 2 replication controller names rather than always taking a file)
+  1. Rollover (replace multiple replication controllers with one, such as to clean up an aborted partial rollout)
+  1. Write a ReplicationController generator to derive the new ReplicationController from an old one (e.g., `--image-version=newversion`, which would apply a name suffix, update a label value, and apply an image tag)
+  1. Use readiness [#620](https://github.com/GoogleCloudPlatform/kubernetes/issues/620)
+  1. Perhaps factor this in a way that it can be shared with [Openshift’s deployment controller](https://github.com/GoogleCloudPlatform/kubernetes/issues/1743) 
+  1. Rolling update service as a plugin
+1. Kind-based filtering on object streams -- only operate on the kinds of objects specified. This would make directory-based kubectl operations much more useful. Users should be able to instantiate the example applications using `kubectl create -f <example-dir> ...`
+1. Improved pretty printing of endpoints, such as in the case that there are more than a few endpoints
+1. Service address/port lookup command(s)
+1. List supported resources
+1. Swagger lookups [#3060](https://github.com/GoogleCloudPlatform/kubernetes/issues/3060)
+1. --name, --name-suffix applied during creation and updates
+1. --labels and opinionated label injection: --app=foo, --tier={fe,cache,be,db}, --uservice=redis, --env={dev,test,prod}, --stage={canary,final}, --track={hourly,daily,weekly}, --release=0.4.3c2. Exact ones TBD. We could allow arbitrary values -- the keys are important. The actual label keys would be (optionally?) namespaced with kubectl.kubernetes.io/, or perhaps the user’s namespace.
+1. --annotations and opinionated annotation injection: --description, --revision
+1. Imperative updates. We'll want to optionally make these safe(r) by supporting preconditions based on the current value and resourceVersion.
+  1. annotation updates similar to label updates
+  1. other custom commands for common imperative updates
+  1. more user-friendly (but still generic) on-command-line json for patch
+1. We also want to support the following flavors of more general updates:
+  1. whichever we don’t support:
+    1. safe update: update the full resource, guarded by resourceVersion precondition (and perhaps selected value-based preconditions)
+    1. forced update: update the full resource, blowing away the previous Spec without preconditions; delete and re-create if necessary
+  1. diff/dryrun: Compare new config with current Spec [#6284](https://github.com/GoogleCloudPlatform/kubernetes/issues/6284)
+  1. submit/apply/reconcile/ensure/merge: Merge user-provided fields with current Spec. Keep track of user-provided fields using an annotation -- see [#1702](https://github.com/GoogleCloudPlatform/kubernetes/issues/1702). Delete all objects with deployment-specific labels.
+1. --dry-run for all commands
+1. Support full label selection syntax, including support for namespaces.
+1. Wait on conditions [#1899](https://github.com/GoogleCloudPlatform/kubernetes/issues/1899)
+1. Make kubectl scriptable: make output and exit code behavior consistent and useful for wrapping in workflows and piping back into kubectl and/or xargs (e.g., dump full URLs?, distinguish permanent and retry-able failure, identify objects that should be retried)
+  1. Here's [an example](http://techoverflow.net/blog/2013/10/22/docker-remove-all-images-and-containers/) where multiple objects on the command line and an option to dump object names only (`-q`) would be useful in combination. [#5906](https://github.com/GoogleCloudPlatform/kubernetes/issues/5906)
+1. Easy generation of clean configuration files from existing objects (including containers -- podex) -- remove readonly fields, status
+  1. Export from one namespace, import into another is an important use case
+1. Derive objects from other objects
+  1. pod clone
+  1. rc from pod
+  1. --labels-from (services from pods or rcs)
+1. Kind discovery (i.e., operate on objects of all kinds) [#5278](https://github.com/GoogleCloudPlatform/kubernetes/issues/5278)
+1. A fairly general-purpose way to specify fields on the command line during creation and update, not just from a config file
+1. Extensible API-based generator framework (i.e. invoke generators via an API/URL rather than building them into kubectl), so that complex client libraries don’t need to be rewritten in multiple languages, and so that the abstractions are available through all interfaces: API, CLI, UI, logs, ... [#5280](https://github.com/GoogleCloudPlatform/kubernetes/issues/5280)
+  1. Need schema registry, and some way to invoke generator (e.g., using a container)
+  1. Convert run command to API-based generator
+1. Transformation framework
+  1. More intelligent defaulting of fields (e.g., [#2643](https://github.com/GoogleCloudPlatform/kubernetes/issues/2643))
+1. Update preconditions based on the values of arbitrary object fields. 
+1. Deployment manager compatibility on GCP: [#3685](https://github.com/GoogleCloudPlatform/kubernetes/issues/3685)
+1. Describe multiple objects, multiple kinds of objects [#5905](https://github.com/GoogleCloudPlatform/kubernetes/issues/5905)
+1. Support yaml document separator [#5840](https://github.com/GoogleCloudPlatform/kubernetes/issues/5840)
+
+TODO: 
+* watch
+* attach [#1521](https://github.com/GoogleCloudPlatform/kubernetes/issues/1521)
+* image/registry commands
+* do any other server paths make sense? validate? generic curl functionality?
+* template parameterization
+* dynamic/runtime configuration
+
+Server-side support:
+
+1. Default selectors from labels [#1698](https://github.com/GoogleCloudPlatform/kubernetes/issues/1698#issuecomment-71048278)
+1. Stop [#1535](https://github.com/GoogleCloudPlatform/kubernetes/issues/1535)
+1. Deleted objects [#2789](https://github.com/GoogleCloudPlatform/kubernetes/issues/2789)
+1. Clone [#170](https://github.com/GoogleCloudPlatform/kubernetes/issues/170)
+1. Resize [#1629](https://github.com/GoogleCloudPlatform/kubernetes/issues/1629)
+1. Useful /operations API: wait for finalization/reification
+1. List supported resources [#2057](https://github.com/GoogleCloudPlatform/kubernetes/issues/2057)
+1. Reverse label lookup [#1348](https://github.com/GoogleCloudPlatform/kubernetes/issues/1348)
+1. Field selection [#1362](https://github.com/GoogleCloudPlatform/kubernetes/issues/1362)
+1. Field filtering [#1459](https://github.com/GoogleCloudPlatform/kubernetes/issues/1459)
+1. Operate on uids
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/cli-roadmap.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/cli-roadmap.md?pixel)]()

release-0.19.0/docs/client-libraries.md

+## kubernetes API client libraries
+
+### Supported
+   * [Go](https://github.com/GoogleCloudPlatform/kubernetes/tree/master/pkg/client)
+
+### User Contributed
+*Note: Libraries provided by outside parties are supported by their authors, not the core Kubernetes team*
+
+   * [Java](https://github.com/fabric8io/fabric8/tree/master/components/kubernetes-api)
+   * [Ruby1](https://github.com/Ch00k/kuber)
+   * [Ruby2](https://github.com/abonas/kubeclient)
+   * [PHP](https://github.com/devstub/kubernetes-api-php-client)
+   * [Node.js](https://github.com/tenxcloud/node-kubernetes-client)
+
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/client-libraries.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/client-libraries.md?pixel)]()

release-0.19.0/docs/cluster-admin-guide.md

+# Kubernetes Cluster Admin Guide
+
+The cluster admin guide is for anyone creating or administering a Kubernetes cluster.
+It assumes some familiarity with concepts in the [User Guide](user-guide.md).
+
+## Planning a cluster
+
+There are many different examples of how to setup a kubernetes cluster.  Many of them are listed in this
+[matrix](getting-started-guides/README.md).  We call each of the combinations in this matrix a *distro*.
+
+Before chosing a particular guide, here are some things to consider:
+ - Are you just looking to try out Kubernetes on your laptop, or build a high-availability many-node cluster? Both
+   models are supported, but some distros are better for one case or the other.
+ - Will you be using a hosted Kubernetes cluster, such as [GKE](https://cloud.google.com/container-engine), or setting
+   one up yourself?
+ - Will your cluster be on-premises, or in the cloud (IaaS)?  Kubernetes does not directly support hybrid clusters.  We
+   recommend setting up multiple clusters rather than spanning distant locations.
+ - Will you be running Kubernetes on "bare metal" or virtual machines?  Kubernetes supports both, via different distros.
+ - Do you just want to run a cluster, or do you expect to do active development of kubernetes project code?  If the
+   latter, it is better to pick a distro actively used by other developers.  Some distros only use binary releases, but
+   offer is a greater variety of choices.
+ - Not all distros are maintained as actively.  Prefer ones which are listed as tested on a more recent version of
+   Kubernetes.
+ - If you are configuring kubernetes on-premises, you will need to consider what [networking
+   model](networking.md) fits best.
+ - If you are designing for very [high-availability](availability.md), you may want multiple clusters in multiple zones.
+
+## Setting up a cluster
+
+Pick one of the Getting Started Guides from the [matrix](getting-started-guides/README.md) and follow it.
+If none of the Getting Started Guides fits, you may want to pull ideas from several of the guides.
+
+One option for custom networking is *OpenVSwitch GRE/VxLAN networking* ([ovs-networking.md](ovs-networking.md)), which
+uses OpenVSwitch to set up networking between pods across
+  Kubernetes nodes.
+
+If you are modifying an existing guide which uses Salt, this document explains [how Salt is used in the Kubernetes
+project.](salt.md).
+
+## Upgrading a cluster
+[Upgrading a cluster](cluster_management.md).
+
+## Managing nodes
+
+[Managing nodes](node.md).
+
+## Optional Cluster Services
+
+* **DNS Integration with SkyDNS** ([dns.md](dns.md)):
+  Resolving a DNS name directly to a Kubernetes service.
+
+* **Logging** with [Kibana](logging.md)
+
+## Multi-tenant support
+
+* **Namespaces** ([namespaces.md](namespaces.md)): Namespaces help different
+  projects, teams, or customers to share a kubernetes cluster.
+
+* **Resource Quota** ([resource_quota_admin.md](resource_quota_admin.md)) 
+
+## Security
+
+* **Kubernetes Container Environment** ([container-environment.md](container-environment.md)):
+  Describes the environment for Kubelet managed containers on a Kubernetes
+  node.
+
+* **Securing access to the API Server** [accessing the api]( accessing_the_api.md)
+
+* **Authentication**  [authentication]( authentication.md)
+
+* **Authorization** [authorization]( authorization.md)
+
+* **Admission Controllers** [admission_controllers]( admission_controllers.md)
+
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/cluster-admin-guide.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/cluster-admin-guide.md?pixel)]()

release-0.19.0/docs/cluster_management.md

+# Cluster Management
+
+This doc is in progress.
+
+## Upgrading a cluster
+
+The `cluster/kube-push.sh` script will do a rudimentary update; it is a 1.0 roadmap item to have a robust live cluster update system.
+
+## Updgrading to a different API version
+
+There is a sequence of steps to upgrade to a new API version.
+
+1. Turn on the new api version
+2. Upgrade the cluster's storage to use the new version.
+3. Upgrade all config files. Identify users of the old api version endpoints.
+4. Update existing objects in the storage to new version by running cluster/update-storage-objects.sh
+3. Turn off the old version.
+
+### Turn on or off an API version for your cluster
+
+Specific API versions can be turned on or off by passing --runtime-config=api/<version> flag while bringing up the server. For example: to turn off v1 API, pass --runtime-config=api/v1=false.
+runtime-config also supports 2 special keys: api/all and api/legacy to control all and legacy APIs respectively. For example, for turning off all api versions except v1, pass --runtime-config=api/all=false,api/v1=true.
+
+### Switching your cluster's storage API version
+
+KUBE_API_VERSIONS env var controls the API versions that are supported in the cluster. The first version in the list is used as the cluster's storage version. Hence, to set a specific version as the storage version, bring it to the front of list of versions in the value of KUBE_API_VERSIONS.
+
+### Switching your config files to a new API version
+
+You can use the kube-version-change utility to convert config files between different API versions.
+
+```
+$ hack/build-go.sh cmd/kube-version-change
+$ _output/local/go/bin/kube-version-change -i myPod.v1beta3.yaml -o myPod.v1.yaml
+```
+
+### Maintenance on a Node
+
+If you need to reboot a node (such as for a kernel upgrade, libc upgrade, hardware repair, etc.), and the downtime is
+brief, then when the Kubelet restarts, it will attempt to restart the pods scheduled to it.  If the reboot takes longer,
+then the node controller will terminate the pods that are bound to the unavailable node.  If there is a corresponding
+replication controller, then a new copy of the pod will be started on a different node.  So, in the case where all
+pods are replicated, upgrades can be done without special coordination.
+
+If you want more control over the upgrading process, you may use the following workflow:
+  1. Mark the node to be rebooted as unschedulable:
+    `kubectl update nodes $NODENAME --patch='{"apiVersion": "v1", "spec": {"unschedulable": true}}'`. 
+    This keeps new pods from landing on the node while you are trying to get them off.
+  1. Get the pods off the machine, via any of the following strategies:
+    1. wait for finite-duration pods to complete
+    1. delete pods with `kubectl delete pods $PODNAME`
+    1. for pods with a replication controller, the pod will eventually be replaced by a new pod which will be scheduled to a new node. additionally, if the pod is part of a service, then clients will automatically be redirected to the new pod.
+    1. for pods with no replication controller, you need to bring up a new copy of the pod, and assuming it is not part of a service, redirect clients to it.
+  1. Work on the node
+  1. Make the node schedulable again:
+    `kubectl update nodes $NODENAME --patch='{"apiVersion": "v1", "spec": {"unschedulable": false}}'`.  
+    If you deleted the node's VM instance and created a new one, then a new schedulable node resource will
+    be created automatically when you create a new VM instance (if you're using a cloud provider that supports
+    node discovery; currently this is only GCE, not including CoreOS on GCE using kube-register). See [Node](node.md).
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/cluster_management.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/cluster_management.md?pixel)]()

release-0.19.0/docs/container-environment.md

+
+# Kubernetes Container Environment
+
+## Overview
+This document describes the environment for Kubelet managed containers on a Kubernetes node (kNode).  In contrast to the Kubernetes cluster API, which provides an API for creating and managing containers, the Kubernetes container environment provides the container access to information about what else is going on in the cluster. 
+
+This cluster information makes it possible to build applications that are *cluster aware*.  
+Additionally, the Kubernetes container environment defines a series of hooks that are surfaced to optional hook handlers defined as part of individual containers.  Container hooks are somewhat analogous to operating system signals in a traditional process model.   However these hooks are designed to make it easier to build reliable, scalable cloud applications in the Kubernetes cluster.  Containers that participate in this cluster lifecycle become *cluster native*. 
+
+Another important part of the container environment is the file system that is available to the container.  In Kubernetes, the filesystem is a combination of an [image](./images.md) and one or more [volumes](./volumes.md).
+
+
+The following sections describe both the cluster information provided to containers, as well as the hooks and life-cycle that allows containers to interact with the management system.
+
+## Cluster Information
+There are two types of information that are available within the container environment.  There is information about the container itself, and there is information about other objects in the system.
+
+### Container Information
+Currently, the only information about the container that is available to the container is the Pod name for the pod in which the container is running.  This ID is set as the hostname of the container, and is accessible through all calls to access the hostname within the container (e.g. the hostname command, or the [gethostname][1] function call in libc).  Additionally, user-defined environment variables from the pod definition, are also available to the container, as are any environment variables specified statically in the Docker image.
+
+In the future, we anticipate expanding this information with richer information about the container.  Examples include available memory, number of restarts, and in general any state that you could get from the call to GET /pods on the API server.
+
+### Cluster Information
+Currently the list of all services that are running at the time when the container was created via the Kubernetes Cluster API are available to the container as environment variables.  The set of environment variables matches the syntax of Docker links.
+
+For a service named **foo** that maps to a container port named **bar**, the following variables are defined:
+
+```sh
+FOO_SERVICE_HOST=<the host the service is running on>
+FOO_SERVICE_PORT=<the port the service is running on>
+```
+
+Going forward, we expect that Services will have a dedicated IP address.  In that context, we will also surface services to the container via DNS.  Of course DNS is still not an enumerable protocol, so we will continue to provide environment variables so that containers can do discovery.
+
+## Container Hooks
+*NB*: Container hooks are under active development, we anticipate adding additional hooks as the Kubernetes container management system evolves.*
+
+Container hooks provide information to the container about events in its management lifecycle.  For example, immediately after a container is started, it receives a *PostStart* hook.  These hooks are broadcast *into* the container with information about the life-cycle of the container.  They are different from the events provided by Docker and other systems which are *output* from the container.  Output events provide a log of what has already happened.  Input hooks provide real-time notification about things that are happening, but no historical log.  
+
+### Hook Details
+There are currently two container hooks that are surfaced to containers, and two proposed hooks:
+
+*PreStart - ****Proposed***
+
+This hook is sent immediately before a container is created.  It notifies that the container will be created immediately after the call completes.  No parameters are passed. *Note - *Some event handlers (namely ‘exec’ are incompatible with this event)
+
+*PostStart*
+
+This hook is sent immediately after a container is created.  It notifies the container that it has been created.  No parameters are passed to the handler.
+
+*PostRestart - ****Proposed***
+
+This hook is called before the PostStart handler, when a container has been restarted, rather than started for the first time.  No parameters are passed to the handler.
+
+*PreStop*
+
+This hook is called immediately before a container is terminated.  This event handler is blocking, and must complete before the call to delete the container is sent to the Docker daemon.  The SIGTERM notification sent by Docker is also still sent.
+
+A single parameter named reason is passed to the handler which contains the reason for termination.  Currently the valid values for reason are:
+
+* ```Delete``` - indicating an API call to delete the pod containing this container.
+* ```Health``` - indicating that a health check of the container failed.
+* ```Dependency``` - indicating that a dependency for the container or the pod is missing, and thus, the container needs to be restarted.  Examples include, the pod infra container crashing, or persistent disk failing for a container that mounts PD.
+
+Eventually, user specified reasons may be [added to the API](https://github.com/GoogleCloudPlatform/kubernetes/issues/137).
+
+
+### Hook Handler Execution
+When a management hook occurs, the management system calls into any registered hook handlers in the container for that hook.  These hook handler calls are synchronous in the context of the pod containing the container. Note:this means that hook handler execution blocks any further management of the pod.  If your hook handler blocks, no other management (including health checks) will occur until the hook handler completes.  Blocking hook handlers do *not* affect management of other Pods.  Typically we expect that users will make their hook handlers as lightweight as possible, but there are cases where long running commands make sense (e.g. saving state prior to container stop)
+
+For hooks which have parameters, these parameters are passed to the event handler as a set of key/value pairs.  The details of this parameter passing is handler implementation dependent (see below).
+
+### Hook delivery guarantees
+Hook delivery is "at least one", which means that a hook may be called multiple times for any given event (e.g. "start" or "stop") and it is up to the hook implementer to be able to handle this
+correctly.
+
+We expect double delivery to be rare, but in some cases if the ```kubelet``` restarts in the middle of sending a hook, the hook may be resent after the kubelet comes back up.
+
+Likewise, we only make a single delivery attempt.  If (for example) an http hook receiver is down, and unable to take traffic, we do not make any attempts to resend.
+
+### Hook Handler Implementations
+Hook handlers are the way that hooks are surfaced to containers.  Containers can select the type of hook handler they would like to implement.  Kubernetes currently supports two different hook handler types:
+
+   * Exec - Executes a specific command (e.g. pre-stop.sh) inside the cgroup and namespaces of the container.  Resources consumed by the command are counted against the container.  Commands which print "ok" to standard out (stdout) are treated as healthy, any other output is treated as container failures (and will cause kubelet to forcibly restart the container).  Parameters are passed to the command as traditional linux command line flags (e.g. pre-stop.sh --reason=HEALTH)
+
+   * HTTP - Executes an HTTP request against a specific endpoint on the container.  HTTP error codes (5xx) and non-response/failure to connect are treated as container failures. Parameters are passed to the http endpoint as query args (e.g. http://some.server.com/some/path?reason=HEALTH)
+
+[1]: http://man7.org/linux/man-pages/man2/gethostname.2.html
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/container-environment.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/container-environment.md?pixel)]()

release-0.19.0/docs/containers.md

+# Containers with Kubernetes
+
+## Containers and commands
+
+So far the Pods we've seen have all used the `image` field to indicate what process Kubernetes
+should run in a container.  In this case, Kubernetes runs the image's default command.  If we want
+to run a particular command or override the image's defaults, there are two additional fields that
+we can use:
+
+1.  `Command`: Controls the actual command run by the image
+2.  `Args`: Controls the arguments passed to the command
+
+### How docker handles command and arguments
+
+Docker images have metadata associated with them that is used to store information about the image.
+The image author may use this to define defaults for the command and arguments to run a container
+when the user does not supply values.  Docker calls the fields for commands and arguments
+`Entrypoint` and `Cmd` respectively.  The full details for this feature are too complicated to 
+describe here, mostly due to the fact that the docker API allows users to specify both of these
+fields as either a string array or a string and there are subtle differences in how those cases are
+handled.  We encourage the curious to check out [docker's documentation]() for this feature.
+
+Kubernetes allows you to override both the image's default command (docker `Entrypoint`) and args
+(docker `Cmd`) with the `Command` and `Args` fields of `Container`.  The rules are:
+
+1.  If you do not supply a `Command` or `Args` for a container, the defaults defined by the image
+    will be used
+2.  If you supply a `Command` but no `Args` for a container, only the supplied `Command` will be
+    used; the image's default arguments are ignored
+3.  If you supply only `Args`, the image's default command will be used with the arguments you
+    supply
+4.  If you supply a `Command` **and** `Args`, the image's defaults will be ignored and the values
+    you supply will be used
+
+Here are examples for these rules in table format
+
+| Image `Entrypoint` |    Image `Cmd`   | Container `Command` |  Container `Args`  |    Command Run   |
+|--------------------|------------------|---------------------|--------------------|------------------|
+|     `[/ep-1]`      |   `[foo bar]`    |   <not set>   |   <not set>  | `[ep-1 foo bar]` | 
+|     `[/ep-1]`      |   `[foo bar]`    |      `[/ep-2]`      |   <not set>  |     `[ep-2]`     | 
+|     `[/ep-1]`      |   `[foo bar]`    |   <not set>   |     `[zoo boo]`    | `[ep-1 zoo boo]` | 
+|     `[/ep-1]`      |   `[foo bar]`    |      `[/ep-2]`      |     `[zoo boo]`    | `[ep-2 zoo boo]` | 
+
+
+## Capabilities
+
+By default, Docker containers are "unprivileged" and cannot, for example, run a Docker daemon inside a Docker container. We can have fine grain control over the capabilities using cap-add and cap-drop.More details [here](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration).
+
+The relationship between Docker's capabilities and [Linux capabilities](http://man7.org/linux/man-pages/man7/capabilities.7.html)
+
+| Docker's capabilities | Linux capabilities |
+| ---- | ---- |
+| SETPCAP |  CAP_SETPCAP |
+| SYS_MODULE |  CAP_SYS_MODULE |
+| SYS_RAWIO |  CAP_SYS_RAWIO |
+| SYS_PACCT |  CAP_SYS_PACCT |
+| SYS_ADMIN |  CAP_SYS_ADMIN |
+| SYS_NICE |  CAP_SYS_NICE |
+| SYS_RESOURCE |  CAP_SYS_RESOURCE |
+| SYS_TIME |  CAP_SYS_TIME |
+| SYS_TTY_CONFIG |  CAP_SYS_TTY_CONFIG |
+| MKNOD |  CAP_MKNOD |
+| AUDIT_WRITE |  CAP_AUDIT_WRITE |
+| AUDIT_CONTROL |  CAP_AUDIT_CONTROL |
+| MAC_OVERRIDE |  CAP_MAC_OVERRIDE |
+| MAC_ADMIN |  CAP_MAC_ADMIN |
+| NET_ADMIN |  CAP_NET_ADMIN |
+| SYSLOG |  CAP_SYSLOG |
+| CHOWN |  CAP_CHOWN |
+| NET_RAW |  CAP_NET_RAW |
+| DAC_OVERRIDE |  CAP_DAC_OVERRIDE |
+| FOWNER |  CAP_FOWNER |
+| DAC_READ_SEARCH |  CAP_DAC_READ_SEARCH |
+| FSETID |  CAP_FSETID |
+| KILL |  CAP_KILL |
+| SETGID |  CAP_SETGID |
+| SETUID |  CAP_SETUID |
+| LINUX_IMMUTABLE |  CAP_LINUX_IMMUTABLE |
+| NET_BIND_SERVICE |  CAP_NET_BIND_SERVICE |
+| NET_BROADCAST |  CAP_NET_BROADCAST |
+| IPC_LOCK |  CAP_IPC_LOCK |
+| IPC_OWNER |  CAP_IPC_OWNER |
+| SYS_CHROOT |  CAP_SYS_CHROOT |
+| SYS_PTRACE |  CAP_SYS_PTRACE |
+| SYS_BOOT |  CAP_SYS_BOOT |
+| LEASE |  CAP_LEASE |
+| SETFCAP |  CAP_SETFCAP |
+| WAKE_ALARM |  CAP_WAKE_ALARM |
+| BLOCK_SUSPEND |  CAP_BLOCK_SUSPEND |
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/containers.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/containers.md?pixel)]()

release-0.19.0/docs/design/README.md

+# Kubernetes Design Overview
+
+Kubernetes is a system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications. 
+
+Kubernetes establishes robust declarative primitives for maintaining the desired state requested by the user. We see these primitives as the main value added by Kubernetes. Self-healing mechanisms, such as auto-restarting, re-scheduling, and replicating containers require active controllers, not just imperative orchestration.
+
+Kubernetes is primarily targeted at applications composed of multiple containers, such as elastic, distributed micro-services. It is also designed to facilitate migration of non-containerized application stacks to Kubernetes. It therefore includes abstractions for grouping containers in both loosely coupled and tightly coupled formations, and provides ways for containers to find and communicate with each other in relatively familiar ways.
+
+Kubernetes enables users to ask a cluster to run a set of containers. The system automatically chooses hosts to run those containers on. While Kubernetes's scheduler is currently very simple, we expect it to grow in sophistication over time. Scheduling is a policy-rich, topology-aware, workload-specific function that significantly impacts availability, performance, and capacity. The scheduler needs to take into account individual and collective resource requirements, quality of service requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on. Workload-specific requirements will be exposed through the API as necessary.
+
+Kubernetes is intended to run on a number of cloud providers, as well as on physical hosts.
+
+A single Kubernetes cluster is not intended to span multiple availability zones. Instead, we recommend building a higher-level layer to replicate complete deployments of highly available applications across multiple zones (see [the availability doc](../availability.md) and [cluster federation proposal](../proposals/federation.md) for more details).
+
+Finally, Kubernetes aspires to be an extensible, pluggable, building-block OSS platform and toolkit. Therefore, architecturally, we want Kubernetes to be built as a collection of pluggable components and layers, with the ability to use alternative schedulers, controllers, storage systems, and distribution mechanisms, and we're evolving its current code in that direction. Furthermore, we want others to be able to extend Kubernetes functionality, such as with higher-level PaaS functionality or multi-cluster layers, without modification of core Kubernetes source. Therefore, its API isn't just (or even necessarily mainly) targeted at end users, but at tool and extension developers. Its APIs are intended to serve as the foundation for an open ecosystem of tools, automation systems, and higher-level API layers. Consequently, there are no "internal" inter-component APIs. All APIs are visible and available, including the APIs used by the scheduler, the node controller, the replication-controller manager, Kubelet's API, etc. There's no glass to break -- in order to handle more complex use cases, one can just access the lower-level APIs in a fully transparent, composable manner.
+
+For more about the Kubernetes architecture, see [architecture](architecture.md).
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/README.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/README.md?pixel)]()

release-0.19.0/docs/design/admission_control.md

+# Kubernetes Proposal - Admission Control
+
+**Related PR:**
+
+| Topic | Link |
+| ----- | ---- |
+| Separate validation from RESTStorage | https://github.com/GoogleCloudPlatform/kubernetes/issues/2977 |
+
+## Background
+
+High level goals:
+
+* Enable an easy-to-use mechanism to provide admission control to cluster
+* Enable a provider to support multiple admission control strategies or author their own
+* Ensure any rejected request can propagate errors back to the caller with why the request failed
+
+Authorization via policy is focused on answering if a user is authorized to perform an action.
+
+Admission Control is focused on if the system will accept an authorized action.
+
+Kubernetes may choose to dismiss an authorized action based on any number of admission control strategies.
+
+This proposal documents the basic design, and describes how any number of admission control plug-ins could be injected.
+
+Implementation of specific admission control strategies are handled in separate documents.
+
+## kube-apiserver
+
+The kube-apiserver takes the following OPTIONAL arguments to enable admission control
+
+| Option | Behavior |
+| ------ | -------- |
+| admission_control | Comma-delimited, ordered list of admission control choices to invoke prior to modifying or deleting an object. |
+| admission_control_config_file | File with admission control configuration parameters to boot-strap plug-in. |
+
+An **AdmissionControl** plug-in is an implementation of the following interface:
+
+```go
+package admission
+
+// Attributes is an interface used by a plug-in to make an admission decision on a individual request.
+type Attributes interface {
+  GetNamespace() string
+  GetKind() string
+  GetOperation() string
+  GetObject() runtime.Object
+}
+
+// Interface is an abstract, pluggable interface for Admission Control decisions.
+type Interface interface {
+  // Admit makes an admission decision based on the request attributes
+  // An error is returned if it denies the request.
+  Admit(a Attributes) (err error)
+}
+```
+
+A **plug-in** must be compiled with the binary, and is registered as an available option by providing a name, and implementation
+of admission.Interface.
+
+```go
+func init() {
+  admission.RegisterPlugin("AlwaysDeny", func(client client.Interface, config io.Reader) (admission.Interface, error) { return NewAlwaysDeny(), nil })
+}
+```
+
+Invocation of admission control is handled by the **APIServer** and not individual **RESTStorage** implementations.
+
+This design assumes that **Issue 297** is adopted, and as a consequence, the general framework of the APIServer request/response flow
+will ensure the following:
+
+1. Incoming request
+2. Authenticate user
+3. Authorize user
+4. If operation=create|update, then validate(object)
+5. If operation=create|update|delete, then admission.Admit(requestAttributes)
+  a. invoke each admission.Interface object in sequence
+6. Object is persisted
+
+If at any step, there is an error, the request is canceled.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/admission_control.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/admission_control.md?pixel)]()

release-0.19.0/docs/design/admission_control_limit_range.md

+# Admission control plugin: LimitRanger
+
+## Background
+
+This document proposes a system for enforcing min/max limits per resource as part of admission control.
+
+## Model Changes
+
+A new resource, **LimitRange**, is introduced to enumerate min/max limits for a resource type scoped to a
+Kubernetes namespace.
+
+```go
+const (
+  // Limit that applies to all pods in a namespace
+  LimitTypePod string = "Pod"
+  // Limit that applies to all containers in a namespace
+  LimitTypeContainer string = "Container"
+)
+
+// LimitRangeItem defines a min/max usage limit for any resource that matches on kind
+type LimitRangeItem struct {
+  // Type of resource that this limit applies to
+  Type string `json:"type,omitempty"`
+  // Max usage constraints on this kind by resource name
+  Max ResourceList `json:"max,omitempty"`
+  // Min usage constraints on this kind by resource name
+  Min ResourceList `json:"min,omitempty"`
+  // Default usage constraints on this kind by resource name
+  Default ResourceList `json:"default,omitempty"`
+}
+
+// LimitRangeSpec defines a min/max usage limit for resources that match on kind
+type LimitRangeSpec struct {
+  // Limits is the list of LimitRangeItem objects that are enforced
+  Limits []LimitRangeItem `json:"limits"`
+}
+
+// LimitRange sets resource usage limits for each kind of resource in a Namespace
+type LimitRange struct {
+  TypeMeta   `json:",inline"`
+  ObjectMeta `json:"metadata,omitempty"`
+
+  // Spec defines the limits enforced
+  Spec LimitRangeSpec `json:"spec,omitempty"`
+}
+
+// LimitRangeList is a list of LimitRange items.
+type LimitRangeList struct {
+  TypeMeta `json:",inline"`
+  ListMeta `json:"metadata,omitempty"`
+
+  // Items is a list of LimitRange objects
+  Items []LimitRange `json:"items"`
+}
+```
+
+## AdmissionControl plugin: LimitRanger
+
+The **LimitRanger** plug-in introspects all incoming admission requests.
+
+It makes decisions by evaluating the incoming object against all defined **LimitRange** objects in the request context namespace.
+
+The following min/max limits are imposed:
+
+**Type: Container**
+
+| ResourceName | Description |
+| ------------ | ----------- |
+| cpu | Min/Max amount of cpu per container |
+| memory | Min/Max amount of memory per container |
+
+**Type: Pod**
+
+| ResourceName | Description |
+| ------------ | ----------- |
+| cpu | Min/Max amount of cpu per pod |
+| memory | Min/Max amount of memory per pod |
+
+If a resource specifies a default value, it may get applied on the incoming resource.  For example, if a default
+value is provided for container cpu, it is set on the incoming container if and only if the incoming container
+does not specify a resource requirements limit field.
+
+If a resource specifies a min value, it may get applied on the incoming resource.  For example, if a min
+value is provided for container cpu, it is set on the incoming container if and only if the incoming container does
+not specify a resource requirements requests field.
+
+If the incoming object would cause a violation of the enumerated constraints, the request is denied with a set of
+messages explaining what constraints were the source of the denial.
+
+If a constraint is not enumerated by a **LimitRange** it is not tracked.
+
+## kube-apiserver
+
+The server is updated to be aware of **LimitRange** objects.
+
+The constraints are only enforced if the kube-apiserver is started as follows:
+
+```
+$ kube-apiserver -admission_control=LimitRanger
+```
+
+## kubectl
+
+kubectl is modified to support the **LimitRange** resource.
+
+```kubectl describe``` provides a human-readable output of limits.
+
+For example,
+
+```shell
+$ kubectl namespace myspace
+$ kubectl create -f examples/limitrange/limit-range.json
+$ kubectl get limits
+NAME
+limits
+$ kubectl describe limits limits
+Name:           limits
+Type            Resource        Min     Max     Default
+----            --------        ---     ---     ---
+Pod             memory          1Mi     1Gi     -
+Pod             cpu             250m    2       -
+Container       memory          1Mi     1Gi     1Mi
+Container       cpu             250m    250m    250m
+```
+
+## Future Enhancements: Define limits for a particular pod or container.
+
+In the current proposal, the **LimitRangeItem** matches purely on **LimitRangeItem.Type**
+
+It is expected we will want to define limits for particular pods or containers by name/uid and label/field selector.
+
+To make a **LimitRangeItem** more restrictive, we will intend to add these additional restrictions at a future point in time.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/admission_control_limit_range.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/admission_control_limit_range.md?pixel)]()

release-0.19.0/docs/design/admission_control_resource_quota.md

+# Admission control plugin: ResourceQuota
+
+## Background
+
+This document proposes a system for enforcing hard resource usage limits per namespace as part of admission control.
+
+## Model Changes
+
+A new resource, **ResourceQuota**, is introduced to enumerate hard resource limits in a Kubernetes namespace.
+
+A new resource, **ResourceQuotaUsage**, is introduced to support atomic updates of a **ResourceQuota** status.
+
+```go
+// The following identify resource constants for Kubernetes object types
+const (
+  // Pods, number
+  ResourcePods ResourceName = "pods"
+  // Services, number
+  ResourceServices ResourceName = "services"
+  // ReplicationControllers, number
+  ResourceReplicationControllers ResourceName = "replicationcontrollers"
+  // ResourceQuotas, number
+  ResourceQuotas ResourceName = "resourcequotas"
+)
+
+// ResourceQuotaSpec defines the desired hard limits to enforce for Quota
+type ResourceQuotaSpec struct {
+  // Hard is the set of desired hard limits for each named resource
+  Hard ResourceList `json:"hard,omitempty"`
+}
+
+// ResourceQuotaStatus defines the enforced hard limits and observed use
+type ResourceQuotaStatus struct {
+  // Hard is the set of enforced hard limits for each named resource
+  Hard ResourceList `json:"hard,omitempty"`
+  // Used is the current observed total usage of the resource in the namespace
+  Used ResourceList `json:"used,omitempty"`
+}
+
+// ResourceQuota sets aggregate quota restrictions enforced per namespace
+type ResourceQuota struct {
+  TypeMeta   `json:",inline"`
+  ObjectMeta `json:"metadata,omitempty"`
+
+  // Spec defines the desired quota
+  Spec ResourceQuotaSpec `json:"spec,omitempty"`
+
+  // Status defines the actual enforced quota and its current usage
+  Status ResourceQuotaStatus `json:"status,omitempty"`
+}
+
+// ResourceQuotaUsage captures system observed quota status per namespace
+// It is used to enforce atomic updates of a backing ResourceQuota.Status field in storage
+type ResourceQuotaUsage struct {
+  TypeMeta   `json:",inline"`
+  ObjectMeta `json:"metadata,omitempty"`
+
+  // Status defines the actual enforced quota and its current usage
+  Status ResourceQuotaStatus `json:"status,omitempty"`
+}
+
+// ResourceQuotaList is a list of ResourceQuota items
+type ResourceQuotaList struct {
+  TypeMeta `json:",inline"`
+  ListMeta `json:"metadata,omitempty"`
+
+  // Items is a list of ResourceQuota objects
+  Items []ResourceQuota `json:"items"`
+}
+
+```
+
+## AdmissionControl plugin: ResourceQuota
+
+The **ResourceQuota** plug-in introspects all incoming admission requests. 
+
+It makes decisions by evaluating the incoming object against all defined **ResourceQuota.Status.Hard** resource limits in the request
+namespace.  If acceptance of the resource would cause the total usage of a named resource to exceed its hard limit, the request is denied.
+
+The following resource limits are imposed as part of core Kubernetes at the namespace level:
+
+| ResourceName | Description |
+| ------------ | ----------- |
+| cpu | Total cpu usage |
+| memory | Total memory usage |
+| pods | Total number of pods  |
+| services | Total number of services |
+| replicationcontrollers | Total number of replication controllers |
+| resourcequotas | Total number of resource quotas |
+
+Any resource that is not part of core Kubernetes must follow the resource naming convention prescribed by Kubernetes.
+
+This means the resource must have a fully-qualified name (i.e. mycompany.org/shinynewresource)
+
+If the incoming request does not cause the total usage to exceed any of the enumerated hard resource limits, the plug-in will post a
+**ResourceQuotaUsage** document to the server to atomically update the observed usage based on the previously read 
+**ResourceQuota.ResourceVersion**.  This keeps incremental usage atomically consistent, but does introduce a bottleneck (intentionally)
+into the system.
+
+To optimize system performance, it is encouraged that all resource quotas are tracked on the same **ResourceQuota** document.  As a result,
+its encouraged to actually impose a cap on the total number of individual quotas that are tracked in the **Namespace** to 1 by explicitly
+capping it in **ResourceQuota** document.
+
+## kube-apiserver
+
+The server is updated to be aware of **ResourceQuota** objects.
+
+The quota is only enforced if the kube-apiserver is started as follows:
+
+```
+$ kube-apiserver -admission_control=ResourceQuota
+```
+
+## kube-controller-manager
+
+A new controller is defined that runs a synch loop to calculate quota usage across the namespace.
+
+**ResourceQuota** usage is only calculated if a namespace has a **ResourceQuota** object.
+
+If the observed usage is different than the recorded usage, the controller sends a **ResourceQuotaUsage** resource
+to the server to atomically update.
+
+The synchronization loop frequency will control how quickly DELETE actions are recorded in the system and usage is ticked down.
+
+To optimize the synchronization loop, this controller will WATCH on Pod resources to track DELETE events, and in response, recalculate
+usage.  This is because a Pod deletion will have the most impact on observed cpu and memory usage in the system, and we anticipate
+this being the resource most closely running at the prescribed quota limits.
+
+## kubectl
+
+kubectl is modified to support the **ResourceQuota** resource.
+
+```kubectl describe``` provides a human-readable output of quota.
+
+For example,
+
+```
+$ kubectl namespace myspace
+$ kubectl create -f examples/resourcequota/resource-quota.json
+$ kubectl get quota
+NAME
+quota
+$ kubectl describe quota quota
+Name:                   quota
+Resource                Used    Hard
+--------                ----    ----
+cpu                     0m      20
+memory                  0       1Gi
+pods                    5       10
+replicationcontrollers  5       20
+resourcequotas          1       1
+services                3       5
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/admission_control_resource_quota.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/admission_control_resource_quota.md?pixel)]()

release-0.19.0/docs/design/architecture.md

+# Kubernetes architecture
+
+A running Kubernetes cluster contains node agents (kubelet) and master components (APIs, scheduler, etc), on top of a distributed storage solution. This diagram shows our desired eventual state, though we're still working on a few things, like making kubelet itself (all our components, really) run within containers, and making the scheduler 100% pluggable.
+
+![Architecture Diagram](../architecture.png?raw=true "Architecture overview")
+
+## The Kubernetes Node
+
+When looking at the architecture of the system, we'll break it down to services that run on the worker node and services that compose the cluster-level control plane.
+
+The Kubernetes node has the services necessary to run application containers and be managed from the master systems.
+
+Each node runs Docker, of course.  Docker takes care of the details of downloading images and running containers.
+
+### Kubelet
+The **Kubelet** manages [pods](../pods.md) and their containers, their images, their volumes, etc. 
+
+### Kube-Proxy
+
+Each node also runs a simple network proxy and load balancer (see the [services FAQ](https://github.com/GoogleCloudPlatform/kubernetes/wiki/Services-FAQ) for more details).  This reflects `services` (see [the services doc](../services.md) for more details) as defined in the Kubernetes API on each node and can do simple TCP and UDP stream forwarding (round robin) across a set of backends.
+
+Service endpoints are currently found via [DNS](../dns.md) or through environment variables (both [Docker-links-compatible](https://docs.docker.com/userguide/dockerlinks/) and Kubernetes {FOO}_SERVICE_HOST and {FOO}_SERVICE_PORT variables are supported).  These variables resolve to ports managed by the service proxy.
+
+## The Kubernetes Control Plane
+
+The Kubernetes control plane is split into a set of components. Currently they all run on a single _master_ node, but that is expected to change soon in order to support high-availability clusters.  These components work together to provide a unified view of the cluster.
+
+### etcd
+
+All persistent master state is stored in an instance of `etcd`.  This provides a great way to store configuration data reliably.  With `watch` support, coordinating components can be notified very quickly of changes.
+
+### Kubernetes API Server
+
+The apiserver serves up the [Kubernetes API](../api.md). It is intended to be a CRUD-y server, with most/all business logic implemented in separate components or in plug-ins. It mainly processes REST operations, validates them, and updates the corresponding objects in `etcd` (and eventually other stores).
+
+### Scheduler
+
+The scheduler binds unscheduled pods to nodes via the `/binding` API. The scheduler is pluggable, and we expect to support multiple cluster schedulers and even user-provided schedulers in the future.
+
+### Kubernetes Controller Manager Server
+
+All other cluster-level functions are currently performed by the Controller Manager. For instance, `Endpoints` objects are created and updated by the endpoints controller, and nodes are discovered, managed, and monitored by the node controller. These could eventually be split into separate components to make them independently pluggable.
+
+The [`replicationcontroller`](../replication-controller.md) is a mechanism that is layered on top of the simple [`pod`](../pods.md) API. We eventually plan to port it to a generic plug-in mechanism, once one is implemented.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/architecture.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/architecture.md?pixel)]()

release-0.19.0/docs/design/clustering.md

+# Clustering in Kubernetes
+
+
+## Overview
+The term "clustering" refers to the process of having all members of the kubernetes cluster find and trust each other.  There are multiple different ways to achieve clustering with different security and usability profiles.  This document attempts to lay out the user experiences for clustering that Kubernetes aims to address.
+
+Once a cluster is established, the following is true:
+
+1. **Master -> Node**  The master needs to know which nodes can take work and what their current status is wrt capacity.
+  1. **Location** The master knows the name and location of all of the nodes in the cluster.
+	  * For the purposes of this doc, location and name should be enough information so that the master can open a TCP connection to the Node.  Most probably we will make this either an IP address or a DNS name.  It is going to be important to be consistent here (master must be able to reach kubelet on that DNS name) so that we can verify certificates appropriately.
+  2. **Target AuthN** A way to securely talk to the kubelet on that node.  Currently we call out to the kubelet over HTTP.  This should be over HTTPS and the master should know what CA to trust for that node.
+  3. **Caller AuthN/Z** This would be the master verifying itself (and permissions) when calling the node.  Currently, this is only used to collect statistics as authorization isn't critical.  This may change in the future though.
+2. **Node -> Master**  The nodes currently talk to the master to know which pods have been assigned to them and to publish events.
+  1. **Location** The nodes must know where the master is at.
+  2. **Target AuthN** Since the master is assigning work to the nodes, it is critical that they verify whom they are talking to.
+  3. **Caller AuthN/Z** The nodes publish events and so must be authenticated to the master. Ideally this authentication is specific to each node so that authorization can be narrowly scoped.  The details of the work to run (including things like environment variables) might be considered sensitive and should be locked down also.
+
+**Note:** While the description here refers to a singular Master, in the future we should enable multiple Masters operating in an HA mode.  While the "Master" is currently the combination of the API Server, Scheduler and Controller Manager, we will restrict ourselves to thinking about the main API and policy engine -- the API Server.
+
+## Current Implementation
+
+A central authority (generally the master) is responsible for determining the set of machines which are members of the cluster.  Calls to create and remove worker nodes in the cluster are restricted to this single authority, and any other requests to add or remove worker nodes are rejected. (1.i).
+
+Communication from the master to nodes is currently over HTTP and is not secured or authenticated in any way.  (1.ii, 1.iii).
+
+The location of the master is communicated out of band to the nodes.  For GCE, this is done via Salt.  Other cluster instructions/scripts use other methods. (2.i)
+
+Currently most communication from the node to the master is over HTTP.  When it is done over HTTPS there is currently no verification of the cert of the master (2.ii).
+
+Currently, the node/kubelet is authenticated to the master via a token shared across all nodes.  This token is distributed out of band (using Salt for GCE) and is optional.  If it is not present then the kubelet is unable to publish events to the master. (2.iii)
+
+Our current mix of out of band communication doesn't meet all of our needs from a security point of view and is difficult to set up and configure.
+
+## Proposed Solution
+
+The proposed solution will provide a range of options for setting up and maintaining a secure Kubernetes cluster.  We want to both allow for centrally controlled systems (leveraging pre-existing trust and configuration systems) or more ad-hoc automagic systems that are incredibly easy to set up.
+
+The building blocks of an easier solution:
+
+* **Move to TLS** We will move to using TLS for all intra-cluster communication.  We will explicitly idenitfy the trust chain (the set of trusted CAs) as opposed to trusting the system CAs.  We will also use client certificates for all AuthN.
+* [optional] **API driven CA** Optionally, we will run a CA in the master that will mint certificates for the nodes/kubelets.  There will be pluggable policies that will automatically approve certificate requests here as appropriate.
+  * **CA approval policy**  This is a pluggable policy object that can automatically approve CA signing requests.  Stock policies will include `always-reject`, `queue` and `insecure-always-approve`.  With `queue` there would be an API for evaluating and accepting/rejecting requests.  Cloud providers could implement a policy here that verifies other out of band information and automatically approves/rejects based on other external factors.
+* **Scoped Kubelet Accounts** These accounts are per-minion and (optionally) give a minion permission to register itself.
+	* To start with, we'd have the kubelets generate a cert/account in the form of `kubelet:<host>`.  To start we would then hard code policy such that we give that particular account appropriate permissions.  Over time, we can make the policy engine more generic.
+* [optional] **Bootstrap API endpoint** This is a helper service hosted outside of the Kubernetes cluster that helps with initial discovery of the master.
+
+### Static Clustering
+
+In this sequence diagram there is out of band admin entity that is creating all certificates and distributing them.  It is also making sure that the kubelets know where to find the master.  This provides for a lot of control but is more difficult to set up as lots of information must be communicated outside of Kubernetes.
+
+![Static Sequence Diagram](clustering/static.png)
+
+### Dynamic Clustering
+
+This diagram dynamic clustering using the bootstrap API endpoint.  That API endpoint is used to both find the location of the master and communicate the root CA for the master.
+
+This flow has the admin manually approving the kubelet signing requests.  This is the `queue` policy defined above.This manual intervention could be replaced by code that can verify the signing requests via other means.
+
+![Dynamic Sequence Diagram](clustering/dynamic.png)
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/clustering.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/clustering.md?pixel)]()

release-0.19.0/docs/design/clustering/.gitignore

+DroidSansMono.ttf

release-0.19.0/docs/design/clustering/Dockerfile

+FROM debian:jessie
+
+RUN apt-get update
+RUN apt-get -qy install python-seqdiag make curl
+
+WORKDIR /diagrams
+
+RUN curl -sLo DroidSansMono.ttf https://googlefontdirectory.googlecode.com/hg/apache/droidsansmono/DroidSansMono.ttf
+
+ADD . /diagrams
+
+CMD bash -c 'make >/dev/stderr && tar cf - *.png'
\ No newline at end of file

release-0.19.0/docs/design/clustering/Makefile

+FONT := DroidSansMono.ttf
+
+PNGS := $(patsubst %.seqdiag,%.png,$(wildcard *.seqdiag))
+
+.PHONY: all
+all: $(PNGS)
+
+.PHONY: watch
+watch:
+	fswatch *.seqdiag | xargs -n 1 sh -c "make || true"
+
+$(FONT):
+	curl -sLo $@ https://googlefontdirectory.googlecode.com/hg/apache/droidsansmono/$(FONT)
+
+%.png: %.seqdiag $(FONT)
+	seqdiag --no-transparency -a -f '$(FONT)' $<
+
+# Build the stuff via a docker image
+.PHONY: docker
+docker:
+	docker build -t clustering-seqdiag .
+	docker run --rm clustering-seqdiag | tar xvf -
+
+docker-clean:
+	docker rmi clustering-seqdiag || true
+	docker images -q --filter "dangling=true" | xargs docker rmi
+
+fix-clock-skew:
+	boot2docker ssh sudo date -u -D "%Y%m%d%H%M.%S" --set "$(shell date -u +%Y%m%d%H%M.%S)"

release-0.19.0/docs/design/clustering/README.md

+This directory contains diagrams for the clustering design doc.
+
+This depends on the `seqdiag` [utility](http://blockdiag.com/en/seqdiag/index.html).  Assuming you have a non-borked python install, this should be installable with
+
+```bash
+pip install seqdiag
+```
+
+Just call `make` to regenerate the diagrams.
+
+## Building with Docker
+If you are on a Mac or your pip install is messed up, you can easily build with docker.
+
+```
+make docker
+```
+
+The first run will be slow but things should be fast after that.
+
+To clean up the docker containers that are created (and other cruft that is left around) you can run `make docker-clean`.
+
+If you are using boot2docker and get warnings about clock skew (or if things aren't building for some reason) then you can fix that up with `make fix-clock-skew`.
+
+## Automatically rebuild on file changes
+
+If you have the fswatch utility installed, you can have it monitor the file system and automatically rebuild when files have changed.  Just do a `make watch`.
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/clustering/README.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/clustering/README.md?pixel)]()

release-0.19.0/docs/design/clustering/dynamic.seqdiag

+seqdiag {
+  activation = none;
+
+
+  user[label = "Admin User"];
+  bootstrap[label = "Bootstrap API\nEndpoint"];
+  master;
+  kubelet[stacked];
+
+  user -> bootstrap [label="createCluster", return="cluster ID"];
+  user <-- bootstrap [label="returns\n- bootstrap-cluster-uri"];
+
+  user ->> master [label="start\n- bootstrap-cluster-uri"];
+  master => bootstrap [label="setMaster\n- master-location\n- master-ca"];
+
+  user ->> kubelet [label="start\n- bootstrap-cluster-uri"];
+  kubelet => bootstrap [label="get-master", return="returns\n- master-location\n- master-ca"];
+  kubelet ->> master [label="signCert\n- unsigned-kubelet-cert", return="retuns\n- kubelet-cert"];
+  user => master [label="getSignRequests"];
+  user => master [label="approveSignRequests"];
+  kubelet <<-- master [label="returns\n- kubelet-cert"];
+
+  kubelet => master [label="register\n- kubelet-location"]
+}

release-0.19.0/docs/design/clustering/static.seqdiag

+seqdiag {
+  activation = none;
+
+  admin[label = "Manual Admin"];
+  ca[label = "Manual CA"]
+  master;
+  kubelet[stacked];
+
+  admin => ca [label="create\n- master-cert"];
+  admin ->> master [label="start\n- ca-root\n- master-cert"];
+
+  admin => ca [label="create\n- kubelet-cert"];
+  admin ->> kubelet [label="start\n- ca-root\n- kubelet-cert\n- master-location"];
+
+  kubelet => master [label="register\n- kubelet-location"];
+}

release-0.19.0/docs/design/command_execution_port_forwarding.md

+# Container Command Execution & Port Forwarding in Kubernetes
+
+## Abstract
+
+This describes an approach for providing support for:
+
+- executing commands in containers, with stdin/stdout/stderr streams attached
+- port forwarding to containers
+
+## Background
+
+There are several related issues/PRs:
+
+- [Support attach](https://github.com/GoogleCloudPlatform/kubernetes/issues/1521)
+- [Real container ssh](https://github.com/GoogleCloudPlatform/kubernetes/issues/1513)
+- [Provide easy debug network access to services](https://github.com/GoogleCloudPlatform/kubernetes/issues/1863)
+- [OpenShift container command execution proposal](https://github.com/openshift/origin/pull/576)
+
+## Motivation
+
+Users and administrators are accustomed to being able to access their systems
+via SSH to run remote commands, get shell access, and do port forwarding.
+
+Supporting SSH to containers in Kubernetes is a difficult task. You must
+specify a "user" and a hostname to make an SSH connection, and `sshd` requires
+real users (resolvable by NSS and PAM). Because a container belongs to a pod,
+and the pod belongs to a namespace, you need to specify namespace/pod/container
+to uniquely identify the target container. Unfortunately, a
+namespace/pod/container is not a real user as far as SSH is concerned. Also,
+most Linux systems limit user names to 32 characters, which is unlikely to be
+large enough to contain namespace/pod/container. We could devise some scheme to
+map each namespace/pod/container to a 32-character user name, adding entries to
+`/etc/passwd` (or LDAP, etc.) and keeping those entries fully in sync all the
+time. Alternatively, we could write custom NSS and PAM modules that allow the
+host to resolve a namespace/pod/container to a user without needing to keep
+files or LDAP in sync.
+
+As an alternative to SSH, we are using a multiplexed streaming protocol that
+runs on top of HTTP. There are no requirements about users being real users,
+nor is there any limitation on user name length, as the protocol is under our
+control. The only downside is that standard tooling that expects to use SSH
+won't be able to work with this mechanism, unless adapters can be written.
+
+## Constraints and Assumptions
+
+- SSH support is not currently in scope
+- CGroup confinement is ultimately desired, but implementing that support is not currently in scope
+- SELinux confinement is ultimately desired, but implementing that support is not currently in scope
+
+## Use Cases
+
+- As a user of a Kubernetes cluster, I want to run arbitrary commands in a container, attaching my local stdin/stdout/stderr to the container
+- As a user of a Kubernetes cluster, I want to be able to connect to local ports on my computer and have them forwarded to ports in the container
+
+## Process Flow
+
+### Remote Command Execution Flow
+1. The client connects to the Kubernetes Master to initiate a remote command execution
+request
+2. The Master proxies the request to the Kubelet where the container lives
+3. The Kubelet executes nsenter + the requested command and streams stdin/stdout/stderr back and forth between the client and the container
+
+### Port Forwarding Flow
+1. The client connects to the Kubernetes Master to initiate a remote command execution
+request
+2. The Master proxies the request to the Kubelet where the container lives
+3. The client listens on each specified local port, awaiting local connections
+4. The client connects to one of the local listening ports
+4. The client notifies the Kubelet of the new connection
+5. The Kubelet executes nsenter + socat and streams data back and forth between the client and the port in the container
+
+
+## Design Considerations
+
+### Streaming Protocol
+
+The current multiplexed streaming protocol used is SPDY. This is not the
+long-term desire, however. As soon as there is viable support for HTTP/2 in Go,
+we will switch to that.
+
+### Master as First Level Proxy
+
+Clients should not be allowed to communicate directly with the Kubelet for
+security reasons. Therefore, the Master is currently the only suggested entry
+point to be used for remote command execution and port forwarding. This is not
+necessarily desirable, as it means that all remote command execution and port
+forwarding traffic must travel through the Master, potentially impacting other
+API requests.
+
+In the future, it might make more sense to retrieve an authorization token from
+the Master, and then use that token to initiate a remote command execution or
+port forwarding request with a load balanced proxy service dedicated to this
+functionality. This would keep the streaming traffic out of the Master.
+
+### Kubelet as Backend Proxy
+
+The kubelet is currently responsible for handling remote command execution and
+port forwarding requests. Just like with the Master described above, this means
+that all remote command execution and port forwarding streaming traffic must
+travel through the Kubelet, which could result in a degraded ability to service
+other requests.
+
+In the future, it might make more sense to use a separate service on the node.
+
+Alternatively, we could possibly inject a process into the container that only
+listens for a single request, expose that process's listening port on the node,
+and then issue a redirect to the client such that it would connect to the first
+level proxy, which would then proxy directly to the injected process's exposed
+port. This would minimize the amount of proxying that takes place.
+
+### Scalability
+
+There are at least 2 different ways to execute a command in a container:
+`docker exec` and `nsenter`. While `docker exec` might seem like an easier and
+more obvious choice, it has some drawbacks.
+
+#### `docker exec`
+
+We could expose `docker exec` (i.e. have Docker listen on an exposed TCP port
+on the node), but this would require proxying from the edge and securing the
+Docker API. `docker exec` calls go through the Docker daemon, meaning that all
+stdin/stdout/stderr traffic is proxied through the Daemon, adding an extra hop.
+Additionally, you can't isolate 1 malicious `docker exec` call from normal
+usage, meaning an attacker could initiate a denial of service or other attack
+and take down the Docker daemon, or the node itself.
+
+We expect remote command execution and port forwarding requests to be long
+running and/or high bandwidth operations, and routing all the streaming data
+through the Docker daemon feels like a bottleneck we can avoid.
+
+#### `nsenter`
+
+The implementation currently uses `nsenter` to run commands in containers,
+joining the appropriate container namespaces. `nsenter` runs directly on the
+node and is not proxied through any single daemon process.
+
+### Security
+
+Authentication and authorization hasn't specifically been tested yet with this
+functionality. We need to make sure that users are not allowed to execute
+remote commands or do port forwarding to containers they aren't allowed to
+access.
+
+Additional work is required to ensure that multiple command execution or port forwarding connections from different clients are not able to see each other's data. This can most likely be achieved via SELinux labeling and unique process contexts.
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/command_execution_port_forwarding.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/command_execution_port_forwarding.md?pixel)]()

release-0.19.0/docs/design/event_compression.md

+# Kubernetes Event Compression
+
+This document captures the design of event compression.
+
+
+## Background
+
+Kubernetes components can get into a state where they generate tons of events which are identical except for the timestamp. For example, when pulling a non-existing image, Kubelet will repeatedly generate ```image_not_existing``` and ```container_is_waiting``` events until upstream components correct the image. When this happens, the spam from the repeated events makes the entire event mechanism useless. It also appears to cause memory pressure in etcd (see [#3853](https://github.com/GoogleCloudPlatform/kubernetes/issues/3853)).
+
+## Proposal
+Each binary that generates events (for example, ```kubelet```) should keep track of previously generated events so that it can collapse recurring events into a single event instead of creating a new instance for each new event.
+
+Event compression should be best effort (not guaranteed). Meaning, in the worst case, ```n``` identical (minus timestamp) events may still result in ```n``` event entries.
+
+## Design
+Instead of a single Timestamp, each event object [contains](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/api/types.go#L1111) the following fields:
+ * ```FirstTimestamp util.Time``` 
+   * The date/time of the first occurrence of the event.
+ * ```LastTimestamp util.Time```
+   * The date/time of the most recent occurrence of the event.
+   * On first occurrence, this is equal to the FirstTimestamp.
+ * ```Count int```
+   * The number of occurrences of this event between FirstTimestamp and LastTimestamp
+   * On first occurrence, this is 1.
+
+Each binary that generates events:
+ * Maintains a historical record of previously generated events:
+   * Implmented with ["Least Recently Used Cache"](https://github.com/golang/groupcache/blob/master/lru/lru.go) in [```pkg/client/record/events_cache.go```](https://github.com/GoogleCloudPlatform/kubernetes/tree/master/pkg/client/record/events_cache.go).
+   * The key in the cache is generated from the event object minus timestamps/count/transient fields, specifically the following events fields are used to construct a unique key for an event:
+     * ```event.Source.Component```
+     * ```event.Source.Host```
+     * ```event.InvolvedObject.Kind```
+     * ```event.InvolvedObject.Namespace```
+     * ```event.InvolvedObject.Name```
+     * ```event.InvolvedObject.UID```
+     * ```event.InvolvedObject.APIVersion```
+     * ```event.Reason```
+     * ```event.Message```
+   * The LRU cache is capped at 4096 events. That means if a component (e.g. kubelet) runs for a long period of time and generates tons of unique events, the previously generated events cache will not grow unchecked in memory. Instead, after 4096 unique events are generated, the oldest events are evicted from the cache.
+ * When an event is generated, the previously generated events cache is checked (see [```pkg/client/record/event.go```](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/client/record/event.go)).
+   * If the key for the new event matches the key for a previously generated event (meaning all of the above fields match between the new event and some previously generated event), then the event is considered to be a duplicate and the existing event entry is updated in etcd:
+     * The new PUT (update) event API is called to update the existing event entry in etcd with the new last seen timestamp and count.
+     * The event is also updated in the previously generated events cache with an incremented count, updated last seen timestamp, name, and new resource version (all required to issue a future event update).
+   * If the key for the new event does not match the key for any previously generated event (meaning none of the above fields match between the new event and any previously generated events), then the event is considered to be new/unique and a new event entry is created in etcd:
+     * The usual POST/create event API is called to create a new event entry in etcd.
+     * An entry for the event is also added to the previously generated events cache.
+
+## Issues/Risks
+ * Compression is not guaranteed, because each component keeps track of event history in memory
+   * An application restart causes event history to be cleared, meaning event history is not preserved across application restarts and compression will not occur across component restarts.
+   * Because an LRU cache is used to keep track of previously generated events, if too many unique events are generated, old events will be evicted from the cache, so events will only be compressed until they age out of the events cache, at which point any new instance of the event will cause a new entry to be created in etcd.
+
+## Example
+Sample kubectl output
+```
+FIRSTSEEN                         LASTSEEN                          COUNT               NAME                                          KIND                SUBOBJECT                                REASON              SOURCE                                                  MESSAGE
+Thu, 12 Feb 2015 01:13:02 +0000   Thu, 12 Feb 2015 01:13:02 +0000   1                   kubernetes-minion-4.c.saad-dev-vms.internal   Minion                                                       starting            {kubelet kubernetes-minion-4.c.saad-dev-vms.internal}   Starting kubelet.
+Thu, 12 Feb 2015 01:13:09 +0000   Thu, 12 Feb 2015 01:13:09 +0000   1                   kubernetes-minion-1.c.saad-dev-vms.internal   Minion                                                       starting            {kubelet kubernetes-minion-1.c.saad-dev-vms.internal}   Starting kubelet.
+Thu, 12 Feb 2015 01:13:09 +0000   Thu, 12 Feb 2015 01:13:09 +0000   1                   kubernetes-minion-3.c.saad-dev-vms.internal   Minion                                                       starting            {kubelet kubernetes-minion-3.c.saad-dev-vms.internal}   Starting kubelet.
+Thu, 12 Feb 2015 01:13:09 +0000   Thu, 12 Feb 2015 01:13:09 +0000   1                   kubernetes-minion-2.c.saad-dev-vms.internal   Minion                                                       starting            {kubelet kubernetes-minion-2.c.saad-dev-vms.internal}   Starting kubelet.
+Thu, 12 Feb 2015 01:13:05 +0000   Thu, 12 Feb 2015 01:13:12 +0000   4                   monitoring-influx-grafana-controller-0133o    Pod                                                          failedScheduling    {scheduler }                                            Error scheduling: no minions available to schedule pods
+Thu, 12 Feb 2015 01:13:05 +0000   Thu, 12 Feb 2015 01:13:12 +0000   4                   elasticsearch-logging-controller-fplln        Pod                                                          failedScheduling    {scheduler }                                            Error scheduling: no minions available to schedule pods
+Thu, 12 Feb 2015 01:13:05 +0000   Thu, 12 Feb 2015 01:13:12 +0000   4                   kibana-logging-controller-gziey               Pod                                                          failedScheduling    {scheduler }                                            Error scheduling: no minions available to schedule pods
+Thu, 12 Feb 2015 01:13:05 +0000   Thu, 12 Feb 2015 01:13:12 +0000   4                   skydns-ls6k1                                  Pod                                                          failedScheduling    {scheduler }                                            Error scheduling: no minions available to schedule pods
+Thu, 12 Feb 2015 01:13:05 +0000   Thu, 12 Feb 2015 01:13:12 +0000   4                   monitoring-heapster-controller-oh43e          Pod                                                          failedScheduling    {scheduler }                                            Error scheduling: no minions available to schedule pods
+Thu, 12 Feb 2015 01:13:20 +0000   Thu, 12 Feb 2015 01:13:20 +0000   1                   kibana-logging-controller-gziey               BoundPod            implicitly required container POD        pulled              {kubelet kubernetes-minion-4.c.saad-dev-vms.internal}   Successfully pulled image "kubernetes/pause:latest"
+Thu, 12 Feb 2015 01:13:20 +0000   Thu, 12 Feb 2015 01:13:20 +0000   1                   kibana-logging-controller-gziey               Pod                                                          scheduled           {scheduler }                                            Successfully assigned kibana-logging-controller-gziey to kubernetes-minion-4.c.saad-dev-vms.internal
+
+```
+
+This demonstrates what would have been 20 separate entries (indicating scheduling failure) collapsed/compressed down to 5 entries.
+
+## Related Pull Requests/Issues
+ * Issue [#4073](https://github.com/GoogleCloudPlatform/kubernetes/issues/4073): Compress duplicate events
+ * PR [#4157](https://github.com/GoogleCloudPlatform/kubernetes/issues/4157): Add "Update Event" to Kubernetes API
+ * PR [#4206](https://github.com/GoogleCloudPlatform/kubernetes/issues/4206): Modify Event struct to allow compressing multiple recurring events in to a single event
+ * PR [#4306](https://github.com/GoogleCloudPlatform/kubernetes/issues/4306): Compress recurring events in to a single event to optimize etcd storage
+ * PR [#4444](https://github.com/GoogleCloudPlatform/kubernetes/pull/4444): Switch events history to use LRU cache instead of map
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/event_compression.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/event_compression.md?pixel)]()

release-0.19.0/docs/design/identifiers.md

+# Identifiers and Names in Kubernetes
+
+A summarization of the goals and recommendations for identifiers in Kubernetes.  Described in [GitHub issue #199](https://github.com/GoogleCloudPlatform/kubernetes/issues/199).
+
+
+## Definitions
+
+UID
+: A non-empty, opaque, system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities.
+
+Name
+: A non-empty string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations.
+
+[rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) label (DNS_LABEL)
+: An alphanumeric (a-z, and 0-9) string, with a maximum length of 63 characters, with the '-' character allowed anywhere except the first or last character, suitable for use as a hostname or segment in a domain name
+
+[rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) subdomain (DNS_SUBDOMAIN)
+: One or more lowercase rfc1035/rfc1123 labels separated by '.' with a maximum length of 253 characters
+
+[rfc4122](http://www.ietf.org/rfc/rfc4122.txt) universally unique identifier (UUID)
+: A 128 bit generated value that is extremely unlikely to collide across time and space and requires no central coordination
+
+
+## Objectives for names and UIDs
+
+1. Uniquely identify (via a UID) an object across space and time
+
+2. Uniquely name (via a name) an object across space
+
+3. Provide human-friendly names in API operations and/or configuration files
+
+4. Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects
+
+5. Allow DNS names to be automatically generated for some objects
+
+
+## General design
+
+1. When an object is created via an API, a Name string (a DNS_SUBDOMAIN) must be specified.  Name must be non-empty and unique within the apiserver.  This enables idempotent and space-unique creation operations.  Parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name.  For situations where generating a name is impractical, some or all objects may support a param to auto-generate a name.  Generating random names will defeat idempotency.
+   * Examples: "guestbook.user", "backend-x4eb1"
+
+2. When an object is created via an API, a Namespace string (a DNS_SUBDOMAIN? format TBD via #1114) may be specified.  Depending on the API receiver, namespaces might be validated (e.g. apiserver might ensure that the namespace actually exists).  If a namespace is not specified, one will be assigned by the API receiver.  This assignment policy might vary across API receivers (e.g. apiserver might have a default, kubelet might generate something semi-random).
+   * Example: "api.k8s.example.com"
+
+3. Upon acceptance of an object via an API, the object is assigned a UID (a UUID).  UID must be non-empty and unique across space and time.
+   * Example: "01234567-89ab-cdef-0123-456789abcdef"
+
+
+## Case study: Scheduling a pod
+
+Pods can be placed onto a particular node in a number of ways.  This case
+study demonstrates how the above design can be applied to satisfy the
+objectives.
+
+### A pod scheduled by a user through the apiserver
+
+1. A user submits a pod with Namespace="" and Name="guestbook" to the apiserver.
+
+2. The apiserver validates the input.
+   1. A default Namespace is assigned.
+   2. The pod name must be space-unique within the Namespace.
+   3. Each container within the pod has a name which must be space-unique within the pod.
+
+3. The pod is accepted.
+   1. A new UID is assigned.
+
+4. The pod is bound to a node.
+   1. The kubelet on the node is passed the pod's UID, Namespace, and Name.
+
+5. Kubelet validates the input.
+
+6. Kubelet runs the pod.
+   1. Each container is started up with enough metadata to distinguish the pod from whence it came.
+   2. Each attempt to run a container is assigned a UID (a string) that is unique across time.
+      * This may correspond to Docker's container ID.
+
+### A pod placed by a config file on the node
+
+1. A config file is stored on the node, containing a pod with UID="", Namespace="", and Name="cadvisor".
+
+2. Kubelet validates the input.
+   1. Since UID is not provided, kubelet generates one.
+   2. Since Namespace is not provided, kubelet generates one.
+      1. The generated namespace should be deterministic and cluster-unique for the source, such as a hash of the hostname and file path.
+         * E.g. Namespace="file-f4231812554558a718a01ca942782d81"
+
+3. Kubelet runs the pod.
+   1. Each container is started up with enough metadata to distinguish the pod from whence it came.
+   2. Each attempt to run a container is assigned a UID (a string) that is unique across time.
+      1. This may correspond to Docker's container ID.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/identifiers.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/identifiers.md?pixel)]()

release-0.19.0/docs/design/persistent-storage.md

+# Persistent Storage
+
+This document proposes a model for managing persistent, cluster-scoped storage for applications requiring long lived data.
+
+### tl;dr
+
+Two new API kinds:
+
+A `PersistentVolume` (PV) is a storage resource provisioned by an administrator.  It is analogous to a node.
+
+A `PersistentVolumeClaim` (PVC) is a user's request for a persistent volume to use in a pod. It is analogous to a pod.  
+
+One new system component:
+
+`PersistentVolumeClaimBinder` is a singleton running in master that watches all PersistentVolumeClaims in the system and binds them to the closest matching available PersistentVolume. The volume manager watches the API for newly created volumes to manage. 
+
+One new volume:
+
+`PersistentVolumeClaimVolumeSource` references the user's PVC in the same namespace.  This volume finds the bound PV and mounts that volume for the pod.  A `PersistentVolumeClaimVolumeSource` is, essentially, a wrapper around another type of volume that is owned by someone else (the system).
+
+Kubernetes makes no guarantees at runtime that the underlying storage exists or is available.  High availability is left to the storage provider.
+
+### Goals
+
+* Allow administrators to describe available storage
+* Allow pod authors to discover and request persistent volumes to use with pods
+* Enforce security through access control lists and securing storage to the same namespace as the pod volume
+* Enforce quotas through admission control
+* Enforce scheduler rules by resource counting
+* Ensure developers can rely on storage being available without being closely bound to a particular disk, server, network, or storage device.
+
+
+#### Describe available storage
+
+Cluster administrators use the API to manage *PersistentVolumes*.  A custom store ```NewPersistentVolumeOrderedIndex``` will index volumes by access modes and sort by storage capacity.  The ```PersistentVolumeClaimBinder``` watches for new claims for storage and binds them to an available volume by matching the volume's characteristics (AccessModes and storage size) to the user's request.
+
+PVs are system objects and, thus, have no namespace.
+
+Many means of dynamic provisioning will be eventually be implemented for various storage types. 
+
+
+##### PersistentVolume API
+
+| Action | HTTP Verb | Path | Description |
+| ---- | ---- | ---- | ---- |
+| CREATE | POST | /api/{version}/persistentvolumes/ | Create instance of PersistentVolume |
+| GET | GET | /api/{version}persistentvolumes/{name} | Get instance of PersistentVolume with {name} |
+| UPDATE | PUT | /api/{version}/persistentvolumes/{name} | Update instance of PersistentVolume with {name} |
+| DELETE | DELETE | /api/{version}/persistentvolumes/{name} | Delete instance of PersistentVolume with {name} |
+| LIST | GET | /api/{version}/persistentvolumes | List instances of PersistentVolume |
+| WATCH | GET | /api/{version}/watch/persistentvolumes | Watch for changes to a PersistentVolume |
+
+
+#### Request Storage
+
+Kubernetes users request persistent storage for their pod by creating a ```PersistentVolumeClaim```.  Their request for storage is described by their requirements for resources and mount capabilities.
+
+Requests for volumes are bound to available volumes by the volume manager, if a suitable match is found.  Requests for resources can go unfulfilled.
+
+Users attach their claim to their pod using a new ```PersistentVolumeClaimVolumeSource``` volume source.
+
+
+##### PersistentVolumeClaim API
+
+
+| Action | HTTP Verb | Path | Description |
+| ---- | ---- | ---- | ---- |
+| CREATE | POST | /api/{version}/namespaces/{ns}/persistentvolumeclaims/ | Create instance of PersistentVolumeClaim in namespace {ns} |
+| GET | GET | /api/{version}/namespaces/{ns}/persistentvolumeclaims/{name} | Get instance of PersistentVolumeClaim in namespace {ns} with {name} |
+| UPDATE | PUT | /api/{version}/namespaces/{ns}/persistentvolumeclaims/{name} | Update instance of PersistentVolumeClaim in namespace {ns} with {name} |
+| DELETE | DELETE | /api/{version}/namespaces/{ns}/persistentvolumeclaims/{name} | Delete instance of PersistentVolumeClaim in namespace {ns} with {name} |
+| LIST | GET | /api/{version}/namespaces/{ns}/persistentvolumeclaims | List instances of PersistentVolumeClaim in namespace {ns} |
+| WATCH | GET | /api/{version}/watch/namespaces/{ns}/persistentvolumeclaims | Watch for changes to PersistentVolumeClaim in namespace {ns} |
+
+
+
+#### Scheduling constraints
+
+Scheduling constraints are to be handled similar to pod resource constraints.  Pods will need to be annotated or decorated with the number of resources it requires on a node.  Similarly, a node will need to list how many it has used or available.
+
+TBD
+
+
+#### Events
+
+The implementation of persistent storage will not require events to communicate to the user the state of their claim.  The CLI for bound claims contains a reference to the backing persistent volume.  This is always present in the API and CLI, making an event to communicate the same unnecessary. 
+
+Events that communicate the state of a mounted volume are left to the volume plugins.
+
+
+### Example
+
+#### Admin provisions storage
+
+An administrator provisions storage by posting PVs to the API.  Various way to automate this task can be scripted.  Dynamic provisioning is a future feature that can maintain levels of PVs.
+
+```
+POST:
+
+kind: PersistentVolume
+apiVersion: v1
+metadata:
+  name: pv0001
+spec:
+  capacity:
+    storage: 10
+  persistentDisk:
+    pdName: "abc123"
+    fsType: "ext4"
+
+--------------------------------------------------
+
+kubectl get pv
+
+NAME                LABELS              CAPACITY            ACCESSMODES         STATUS              CLAIM
+pv0001              map[]               10737418240         RWO                 Pending    
+
+
+```
+
+#### Users request storage
+
+A user requests storage by posting a PVC to the API.  Their request contains the AccessModes they wish their volume to have and the minimum size needed.
+
+The user must be within a namespace to create PVCs.
+
+```
+
+POST: 
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: myclaim-1
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 3
+
+--------------------------------------------------
+
+kubectl get pvc
+
+
+NAME                LABELS              STATUS              VOLUME
+myclaim-1           map[]               pending                         
+
+```
+
+
+#### Matching and binding
+
+  The ```PersistentVolumeClaimBinder``` attempts to find an available volume that most closely matches the user's request.  If one exists, they are bound by putting a reference on the PV to the PVC.  Requests can go unfulfilled if a suitable match is not found.
+
+```
+
+kubectl get pv
+
+NAME                LABELS              CAPACITY            ACCESSMODES         STATUS              CLAIM
+pv0001              map[]               10737418240         RWO                 Bound               myclaim-1 / f4b3d283-c0ef-11e4-8be4-80e6500a981e
+
+
+kubectl get pvc
+
+NAME                LABELS              STATUS              VOLUME
+myclaim-1           map[]               Bound               b16e91d6-c0ef-11e4-8be4-80e6500a981e
+
+
+```
+
+#### Claim usage
+
+The claim holder can use their claim as a volume.  The ```PersistentVolumeClaimVolumeSource``` knows to fetch the PV backing the claim and mount its volume for a pod.
+
+The claim holder owns the claim and its data for as long as the claim exists.  The pod using the claim can be deleted, but the claim remains in the user's namespace.  It can be used again and again by many pods.
+
+```
+POST: 
+
+kind: Pod
+apiVersion: v1
+metadata:
+  name: mypod
+spec:
+  containers:
+    - image: nginx
+      name: myfrontend
+      volumeMounts:
+      - mountPath: "/var/www/html"
+        name: mypd
+  volumes:
+    - name: mypd
+      source:
+        persistentVolumeClaim:
+         accessMode: ReadWriteOnce
+         claimRef:
+           name: myclaim-1
+
+```
+
+#### Releasing a claim and Recycling a volume
+
+When a claim holder is finished with their data, they can delete their claim.
+
+```
+
+kubectl delete pvc myclaim-1
+
+```
+
+The ```PersistentVolumeClaimBinder``` will reconcile this by removing the claim reference from the PV and change the PVs status to 'Released'.   
+
+Admins can script the recycling of released volumes.  Future dynamic provisioners will understand how a volume should be recycled.  
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/persistent-storage.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/persistent-storage.md?pixel)]()

release-0.19.0/docs/design/principles.md

+# Design Principles
+
+Principles to follow when extending Kubernetes. 
+
+## API
+
+See also the [API conventions](../api-conventions.md).
+
+* All APIs should be declarative.
+* API objects should be complementary and composable, not opaque wrappers.
+* The control plane should be transparent -- there are no hidden internal APIs.
+* The cost of API operations should be proportional to the number of objects intentionally operated upon. Therefore, common filtered lookups must be indexed. Beware of patterns of multiple API calls that would incur quadratic behavior.
+* Object status must be 100% reconstructable by observation. Any history kept must be just an optimization and not required for correct operation.
+* Cluster-wide invariants are difficult to enforce correctly. Try not to add them. If you must have them, don't enforce them atomically in master components, that is contention-prone and doesn't provide a recovery path in the case of a bug allowing the invariant to be violated. Instead, provide a series of checks to reduce the probability of a violation, and make every component involved able to recover from an invariant violation. 
+* Low-level APIs should be designed for control by higher-level systems. Higher-level APIs should be intent-oriented (think SLOs) rather than implementation-oriented (think control knobs).
+
+## Control logic
+
+* Functionality must be *level-based*, meaning the system must operate correctly given the desired state and the current/observed state, regardless of how many intermediate state updates may have been missed. Edge-triggered behavior must be just an optimization.
+* Assume an open world: continually verify assumptions and gracefully adapt to external events and/or actors. Example: we allow users to kill pods under control of a replication controller; it just replaces them.
+* Do not define comprehensive state machines for objects with behaviors associated with state transitions and/or "assumed" states that cannot be ascertained by observation. 
+* Don't assume a component's decisions will not be overridden or rejected, nor for the component to always understand why. For example, etcd may reject writes. Kubelet may reject pods. The scheduler may not be able to schedule pods. Retry, but back off and/or make alternative decisions.
+* Components should be self-healing. For example, if you must keep some state (e.g., cache) the content needs to be periodically refreshed, so that if an item does get erroneously stored or a deletion event is missed etc, it will be soon fixed, ideally on timescales that are shorter than what will attract attention from humans.
+* Component behavior should degrade gracefully. Prioritize actions so that the most important activities can continue to function even when overloaded and/or in states of partial failure.
+
+## Architecture
+
+* Only the apiserver should communicate with etcd/store, and not other components (scheduler, kubelet, etc.).
+* Compromising a single node shouldn't compromise the cluster.
+* Components should continue to do what they were last told in the absence of new instructions (e.g., due to network partition or component outage).
+* All components should keep all relevant state in memory all the time. The apiserver should write through to etcd/store, other components should write through to the apiserver, and they should watch for updates made by other clients. 
+* Watch is preferred over polling.
+
+## Extensibility
+
+TODO: pluggability
+
+## Bootstrapping
+
+* [Self-hosting](https://github.com/GoogleCloudPlatform/kubernetes/issues/246) of all components is a goal.
+* Minimize the number of dependencies, particularly those required for steady-state operation.
+* Stratify the dependencies that remain via principled layering.
+* Break any circular dependencies by converting hard dependencies to soft dependencies.
+  * Also accept that data from other components from another source, such as local files, which can then be manually populated at bootstrap time and then continuously updated once those other components are available.
+  * State should be rediscoverable and/or reconstructable.
+  * Make it easy to run temporary, bootstrap instances of all components in order to create the runtime state needed to run the components in the steady state; use a lock (master election for distributed components, file lock for local components like Kubelet) to coordinate handoff. We call this technique "pivoting".
+  * Have a solution to restart dead components. For distributed components, replication works well. For local components such as Kubelet, a process manager or even a simple shell loop works.
+
+## Availability
+
+TODO
+
+## General principles
+
+* [Eric Raymond's 17 UNIX rules](https://en.wikipedia.org/wiki/Unix_philosophy#Eric_Raymond.E2.80.99s_17_Unix_Rules)
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/principles.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/principles.md?pixel)]()

release-0.19.0/docs/design/security_context.md

+# Security Contexts
+## Abstract
+A security context is a set of constraints that are applied to a container in order to achieve the following goals (from [security design](security.md)):
+
+1.  Ensure a clear isolation between container and the underlying host it runs on
+2.  Limit the ability of the container to negatively impact the infrastructure or other containers
+
+## Background
+
+The problem of securing containers in Kubernetes has come up [before](https://github.com/GoogleCloudPlatform/kubernetes/issues/398) and the potential problems with container security are [well known](http://opensource.com/business/14/7/docker-security-selinux). Although it is not possible to completely isolate Docker containers from their hosts, new features like [user namespaces](https://github.com/docker/libcontainer/pull/304) make it possible to greatly reduce the attack surface.
+
+## Motivation
+
+### Container isolation
+
+In order to improve container isolation from host and other containers running on the host, containers should only be 
+granted the access they need to perform their work. To this end it should be possible to take advantage of Docker 
+features such as the ability to [add or remove capabilities](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration) and [assign MCS labels](https://docs.docker.com/reference/run/#security-configuration) 
+to the container process.
+
+Support for user namespaces has recently been [merged](https://github.com/docker/libcontainer/pull/304) into Docker's libcontainer project and should soon surface in Docker itself. It will make it possible to assign a range of unprivileged uids and gids from the host to each container, improving the isolation between host and container and between containers.
+
+### External integration with shared storage
+In order to support external integration with shared storage, processes running in a Kubernetes cluster 
+should be able to be uniquely identified by their Unix UID, such that a chain of  ownership can be established. 
+Processes in pods will need to have consistent UID/GID/SELinux category labels in order to access shared disks.
+
+## Constraints and Assumptions
+* It is out of the scope of this document to prescribe a specific set 
+  of constraints to isolate containers from their host. Different use cases need different
+  settings.
+* The concept of a security context should not be tied to a particular security mechanism or platform 
+  (ie. SELinux, AppArmor)
+* Applying a different security context to a scope (namespace or pod) requires a solution such as the one proposed for
+  [service accounts](./service_accounts.md).
+
+## Use Cases
+
+In order of increasing complexity, following are example use cases that would 
+be addressed with security contexts:
+
+1.  Kubernetes is used to run a single cloud application. In order to protect
+    nodes from containers:
+    * All containers run as a single non-root user
+    * Privileged containers are disabled
+    * All containers run with a particular MCS label 
+    * Kernel capabilities like CHOWN and MKNOD are removed from containers
+    
+2.  Just like case #1, except that I have more than one application running on
+    the Kubernetes cluster.
+    * Each application is run in its own namespace to avoid name collisions
+    * For each application a different uid and MCS label is used
+    
+3.  Kubernetes is used as the base for a PAAS with 
+    multiple projects, each project represented by a namespace. 
+    * Each namespace is associated with a range of uids/gids on the node that
+      are mapped to uids/gids on containers using linux user namespaces. 
+    * Certain pods in each namespace have special privileges to perform system
+      actions such as talking back to the server for deployment, run docker
+      builds, etc.
+    * External NFS storage is assigned to each namespace and permissions set
+      using the range of uids/gids assigned to that namespace. 
+
+## Proposed Design
+
+### Overview
+A *security context* consists of a set of constraints that determine how a container
+is secured before getting created and run. A security context resides on the container and represents the runtime parameters that will
+be used to create and run the container via container APIs. A *security context provider* is passed to the Kubelet so it can have a chance
+to mutate Docker API calls in order to apply the security context.
+
+It is recommended that this design be implemented in two phases:
+
+1.  Implement the security context provider extension point in the Kubelet 
+    so that a default security context can be applied on container run and creation.
+2.  Implement a security context structure that is part of a service account. The
+    default context provider can then be used to apply a security context based
+    on the service account associated with the pod.
+    
+### Security Context Provider
+
+The Kubelet will have an interface that points to a `SecurityContextProvider`. The `SecurityContextProvider` is invoked before creating and running a given container:
+
+```go
+type SecurityContextProvider interface {
+	// ModifyContainerConfig is called before the Docker createContainer call.
+	// The security context provider can make changes to the Config with which
+	// the container is created.
+	// An error is returned if it's not possible to secure the container as 
+	// requested with a security context. 
+	ModifyContainerConfig(pod *api.Pod, container *api.Container, config *docker.Config)
+	
+	// ModifyHostConfig is called before the Docker runContainer call.
+	// The security context provider can make changes to the HostConfig, affecting
+	// security options, whether the container is privileged, volume binds, etc.
+	// An error is returned if it's not possible to secure the container as requested 
+	// with a security context. 
+	ModifyHostConfig(pod *api.Pod, container *api.Container, hostConfig *docker.HostConfig)
+}
+```
+
+If the value of the SecurityContextProvider field on the Kubelet is nil, the kubelet will create and run the container as it does today.   
+
+### Security Context
+
+A security context resides on the container and represents the runtime parameters that will
+be used to create and run the container via container APIs. Following is an example of an initial implementation:
+
+```go
+type type Container struct {
+	... other fields omitted ...
+	// Optional: SecurityContext defines the security options the pod should be run with
+    SecurityContext *SecurityContext
+}
+
+// SecurityContext holds security configuration that will be applied to a container.  SecurityContext
+// contains duplication of some existing fields from the Container resource.  These duplicate fields
+// will be populated based on the Container configuration if they are not set.  Defining them on
+// both the Container AND the SecurityContext will result in an error.
+type SecurityContext struct {
+	// Capabilities are the capabilities to add/drop when running the container
+	Capabilities *Capabilities
+
+	// Run the container in privileged mode
+	Privileged *bool
+
+	// SELinuxOptions are the labels to be applied to the container
+	// and volumes
+	SELinuxOptions *SELinuxOptions
+
+	// RunAsUser is the UID to run the entrypoint of the container process.
+	RunAsUser *int64
+}
+
+// SELinuxOptions are the labels to be applied to the container.
+type SELinuxOptions struct {
+	// SELinux user label
+	User string
+
+	// SELinux role label
+	Role string
+
+	// SELinux type label
+	Type string
+
+	// SELinux level label.
+	Level string
+}
+```
+### Admission
+
+It is up to an admission plugin to determine if the security context is acceptable or not.  At the
+time of writing, the admission control plugin for security contexts will only allow a context that
+has defined capabilities or privileged.  Contexts that attempt to define a UID or SELinux options
+will be denied by default.  In the future the admission plugin will base this decision upon
+configurable policies that reside within the [service account](https://github.com/GoogleCloudPlatform/kubernetes/pull/2297).
+
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/security_context.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/security_context.md?pixel)]()

release-0.19.0/docs/design/service_accounts.md

+#Service Accounts
+
+## Motivation
+
+Processes in Pods may need to call the Kubernetes API.  For example:
+  - scheduler
+  - replication controller
+  - minion controller
+  - a map-reduce type framework which has a controller that then tries to make a dynamically determined number of workers and watch them
+  - continuous build and push system
+  - monitoring system
+
+They also may interact with services other than the Kubernetes API, such as:
+  - an image repository, such as docker -- both when the images are pulled to start the containers, and for writing
+    images in the case of pods that generate images.
+  - accessing other cloud services, such as blob storage, in the context of a larged, integrated, cloud offering (hosted
+    or private).
+  - accessing files in an NFS volume attached to the pod
+
+## Design Overview
+A service account binds together several things:
+  - a *name*, understood by users, and perhaps by peripheral systems, for an identity
+  - a *principal* that can be authenticated and [authorized](../authorization.md)
+  - a [security context](./security_context.md), which defines the Linux Capabilities, User IDs, Groups IDs, and other
+    capabilities and controls on interaction with the file system and OS.
+  - a set of [secrets](./secrets.md), which a container may use to
+    access various networked resources.
+
+## Design Discussion
+
+A new object Kind is added:
+```go
+type ServiceAccount struct {
+    TypeMeta   `json:",inline" yaml:",inline"`
+    ObjectMeta `json:"metadata,omitempty" yaml:"metadata,omitempty"`
+
+    username string
+    securityContext ObjectReference // (reference to a securityContext object)
+    secrets []ObjectReference // (references to secret objects
+}
+```
+
+The name ServiceAccount is chosen because it is widely used already (e.g. by Kerberos and LDAP)
+to refer to this type of account.  Note that it has no relation to kubernetes Service objects.
+
+The ServiceAccount object does not include any information that could not be defined separately:
+  - username can be defined however users are defined.
+  - securityContext and secrets are only referenced and are created using the REST API.
+
+The purpose of the serviceAccount object is twofold:
+  - to bind usernames to securityContexts and secrets, so that the username can be used to refer succinctly
+    in contexts where explicitly naming securityContexts and secrets would be inconvenient
+  - to provide an interface to simplify allocation of new securityContexts and secrets.
+These features are explained later.
+
+### Names
+
+From the standpoint of the Kubernetes API, a `user` is any principal which can authenticate to kubernetes API.
+This includes a human running `kubectl` on her desktop and a container in a Pod on a Node making API calls.
+
+There is already a notion of a username in kubernetes, which is populated into a request context after authentication.
+However, there is no API object representing a user.  While this may evolve, it is expected that in mature installations,
+the canonical storage of user identifiers will be handled by a system external to kubernetes.
+
+Kubernetes does not dictate how to divide up the space of user identifier strings.  User names can be
+simple Unix-style short usernames, (e.g. `alice`), or may be qualified to allow for federated identity (
+`alice@example.com` vs `alice@example.org`.)  Naming convention may distinguish service accounts from user
+accounts (e.g. `alice@example.com` vs `build-service-account-a3b7f0@foo-namespace.service-accounts.example.com`),
+but Kubernetes does not require this.
+
+Kubernetes also does not require that there be a distinction between human and Pod users.  It will be possible
+to setup a cluster where Alice the human talks to the kubernetes API as username `alice` and starts pods that
+also talk to the API as user `alice` and write files to NFS as user `alice`.  But, this is not recommended.
+
+Instead, it is recommended that Pods and Humans have distinct identities, and reference implementations will
+make this distinction.
+
+The distinction is useful for a number of reasons:
+  - the requirements for humans and automated processes are different:
+    - Humans need a wide range of capabilities to do their daily activities. Automated processes often have more narrowly-defined activities.
+    - Humans may better tolerate the exceptional conditions created by expiration of a token.  Remembering to handle
+      this in a program is more annoying.  So, either long-lasting credentials or automated rotation of credentials is
+      needed.
+    - A Human typically keeps credentials on a machine that is not part of the cluster and so not subject to automatic
+      management.  A VM with a role/service-account can have its credentials automatically managed.
+  - the identity of a Pod cannot in general be mapped to a single human.
+    - If policy allows, it may be created by one human, and then updated by another, and another, until its behavior cannot be attributed to a single human.
+
+**TODO**: consider getting rid of separate serviceAccount object and just rolling its parts into the SecurityContext or
+Pod Object.
+
+The `secrets` field is a list of references to /secret objects that an process started as that service account should
+have access to to be able to assert that role.
+
+The secrets are not inline with the serviceAccount object.  This way, most or all users can have permission to `GET /serviceAccounts` so they can remind themselves
+what serviceAccounts are available for use.
+
+Nothing will prevent creation of a serviceAccount with two secrets of type `SecretTypeKubernetesAuth`, or secrets of two
+different types.  Kubelet and client libraries will have some behavior, TBD, to handle the case of multiple secrets of a
+given type (pick first or provide all and try each in order, etc).
+
+When a serviceAccount and a matching secret exist, then a `User.Info` for the serviceAccount and a `BearerToken` from the secret
+are added to the map of tokens used by the authentication process in the apiserver, and similarly for other types.  (We
+might have some types that do not do anything on apiserver but just get pushed to the kubelet.)
+
+### Pods
+The `PodSpec` is extended to have a `Pods.Spec.ServiceAccountUsername` field.  If this is unset, then a
+default value is chosen.  If it is set, then the corresponding value of `Pods.Spec.SecurityContext` is set by the
+Service Account Finalizer (see below).
+
+TBD: how policy limits which users can make pods with which service accounts.
+
+### Authorization
+Kubernetes API Authorization Policies refer to users.  Pods created with a `Pods.Spec.ServiceAccountUsername` typically
+get a `Secret` which allows them to authenticate to the Kubernetes APIserver as a particular user.  So any
+policy that is desired can be applied to them.
+
+A higher level workflow is needed to coordinate creation of serviceAccounts, secrets and relevant policy objects.
+Users are free to extend kubernetes to put this business logic wherever is convenient for them, though the
+Service Account Finalizer is one place where this can happen (see below).
+
+### Kubelet
+
+The kubelet will treat as "not ready to run" (needing a finalizer to act on it) any Pod which has an empty
+SecurityContext.
+
+The kubelet will set a default, restrictive, security context for any pods created from non-Apiserver config
+sources (http, file).
+
+Kubelet watches apiserver for secrets which are needed by pods bound to it.
+
+**TODO**: how to only let kubelet see secrets it needs to know.
+
+### The service account finalizer
+
+There are several ways to use Pods with SecurityContexts and Secrets.
+
+One way is to explicitly specify the securityContext and all secrets of a Pod when the pod is initially created,
+like this:
+
+**TODO**: example of pod with explicit refs.
+
+Another way is with the *Service Account Finalizer*, a plugin process which is optional, and which handles
+business logic around service accounts.
+
+The Service Account Finalizer watches Pods, Namespaces, and ServiceAccount definitions.
+
+First, if it finds pods which have a `Pod.Spec.ServiceAccountUsername` but no `Pod.Spec.SecurityContext` set,
+then it copies in the referenced securityContext and secrets references for the corresponding `serviceAccount`.
+
+Second, if ServiceAccount definitions change, it may take some actions.
+**TODO**: decide what actions it takes when a serviceAccount definition changes.  Does it stop pods, or just
+allow someone to list ones that out out of spec?  In general, people may want to customize this?
+
+Third, if a new namespace is created, it may create a new serviceAccount for that namespace.  This may include
+a new username (e.g. `NAMESPACE-default-service-account@serviceaccounts.$CLUSTERID.kubernetes.io`), a new
+securityContext, a newly generated secret to authenticate that serviceAccount to the Kubernetes API, and default
+policies for that service account.
+**TODO**: more concrete example.  What are typical default permissions for default service account (e.g. readonly access
+to services in the same namespace and read-write access to events in that namespace?)
+
+Finally, it may provide an interface to automate creation of new serviceAccounts.  In that case, the user may want
+to GET serviceAccounts to see what has been created.
+
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/service_accounts.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/service_accounts.md?pixel)]()

release-0.19.0/docs/design/simple-rolling-update.md

+## Simple rolling update
+This is a lightweight design document for simple rolling update in ```kubectl```
+
+Complete execution flow can be found [here](#execution-details).
+
+### Lightweight rollout
+Assume that we have a current replication controller named ```foo``` and it is running image ```image:v1```
+
+```kubectl rolling-update rc foo [foo-v2] --image=myimage:v2```
+
+If the user doesn't specify a name for the 'next' controller, then the 'next' controller is renamed to
+the name of the original controller.
+
+Obviously there is a race here, where if you kill the client between delete foo, and creating the new version of 'foo' you might be surprised about what is there, but I think that's ok.
+See [Recovery](#recovery) below
+
+If the user does specify a name for the 'next' controller, then the 'next' controller is retained with its existing name,
+and the old 'foo' controller is deleted.  For the purposes of the rollout, we add a unique-ifying label ```kubernetes.io/deployment``` to both the ```foo``` and ```foo-next``` controllers.
+The value of that label is the hash of the complete JSON representation of the```foo-next``` or```foo``` controller.  The name of this label can be overridden by the user with the ```--deployment-label-key``` flag.
+
+#### Recovery
+If a rollout fails or is terminated in the middle, it is important that the user be able to resume the roll out.
+To facilitate recovery in the case of a crash of the updating process itself, we add the following annotations to each replicaController in the ```kubernetes.io/``` annotation namespace:
+   * ```desired-replicas``` The desired number of replicas for this controller (either N or zero)
+   * ```update-partner``` A pointer to the replicaiton controller resource that is the other half of this update (syntax ```<name>``` the namespace is assumed to be identical to the namespace of this replication controller.)
+
+Recovery is achieved by issuing the same command again:
+
+```
+kubectl rolling-update rc foo [foo-v2] --image=myimage:v2
+```
+
+Whenever the rolling update command executes, the kubectl client looks for replication controllers called ```foo``` and ```foo-next```, if they exist, an attempt is
+made to roll ```foo``` to ```foo-next```.  If ```foo-next``` does not exist, then it is created, and the rollout is a new rollout.  If ```foo``` doesn't exist, then
+it is assumed that the rollout is nearly completed, and ```foo-next``` is renamed to ```foo```.  Details of the execution flow are given below.
+
+
+### Aborting a rollout
+Abort is assumed to want to reverse a rollout in progress.
+
+```kubectl rolling-update rc foo [foo-v2] --rollback```
+
+This is really just semantic sugar for:
+
+```kubectl rolling-update rc foo-v2 foo```
+
+With the added detail that it moves the ```desired-replicas``` annotation from ```foo-v2``` to ```foo```
+
+
+### Execution Details
+
+For the purposes of this example, assume that we are rolling from ```foo``` to ```foo-next``` where the only change is an image update from `v1` to `v2`
+
+If the user doesn't specify a ```foo-next``` name, then it is either discovered from the ```update-partner``` annotation on ```foo```.  If that annotation doesn't exist,
+then ```foo-next``` is synthesized using the pattern ```<controller-name>-<hash-of-next-controller-JSON>```
+
+#### Initialization
+   * If ```foo``` and ```foo-next``` do not exist:
+      * Exit, and indicate an error to the user, that the specified controller doesn't exist.
+   * If ```foo``` exists, but ```foo-next``` does not:
+      * Create ```foo-next``` populate it with the ```v2``` image, set ```desired-replicas``` to ```foo.Spec.Replicas```
+      * Goto Rollout
+   * If ```foo-next``` exists, but ```foo``` does not:
+      * Assume that we are in the rename phase.
+      * Goto Rename
+   * If both ```foo``` and ```foo-next``` exist:
+      * Assume that we are in a partial rollout
+      * If ```foo-next``` is missing the ```desired-replicas``` annotation
+         * Populate the ```desired-replicas``` annotation to ```foo-next``` using the current size of ```foo```
+      * Goto Rollout
+
+#### Rollout
+   * While size of ```foo-next``` < ```desired-replicas``` annotation on ```foo-next```
+      * increase size of ```foo-next```
+      * if size of ```foo``` > 0
+         decrease size of ```foo```
+   * Goto Rename
+
+#### Rename
+   * delete ```foo```
+   * create ```foo``` that is identical to ```foo-next```
+   * delete ```foo-next```
+
+#### Abort
+   * If ```foo-next``` doesn't exist
+      * Exit and indicate to the user that they may want to simply do a new rollout with the old version
+   * If ```foo``` doesn't exist
+      * Exit and indicate not found to the user
+   * Otherwise, ```foo-next``` and ```foo``` both exist
+      * Set ```desired-replicas``` annotation on ```foo``` to match the annotation on ```foo-next```
+      * Goto Rollout with ```foo``` and ```foo-next``` trading places.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/design/simple-rolling-update.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/design/simple-rolling-update.md?pixel)]()

release-0.19.0/docs/devel/README.md

+# Developing Kubernetes
+
+Docs in this directory relate to developing Kubernetes.
+
+* **On Collaborative Development** ([collab.md](collab.md)): info on pull requests and code reviews.
+
+* **Development Guide** ([development.md](development.md)): Setting up your environment tests.
+
+* **Hunting flaky tests** ([flaky-tests.md](flaky-tests.md)): We have a goal of 99.9% flake free tests.
+  Here's how to run your tests many times.
+
+* **GitHub Issues** ([issues.md](issues.md)): How incoming issues are reviewed and prioritized.
+
+* **Logging Conventions** ([logging.md](logging.md)]: Glog levels.
+
+* **Pull Request Process** ([pull-requests.md](pull-requests.md)): When and why pull requests are closed.
+
+* **Releasing Kubernetes** ([releasing.md](releasing.md)): How to create a Kubernetes release (as in version)
+  and how the version information gets embedded into the built binaries.
+
+* **Profiling Kubernetes** ([profiling.md](profiling.md)): How to plug in go pprof profiler to Kubernetes.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/README.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/README.md?pixel)]()

release-0.19.0/docs/devel/coding-conventions.md

+Coding style advice for contributors
+  - Bash
+    - https://google-styleguide.googlecode.com/svn/trunk/shell.xml
+  - Go
+    - https://github.com/golang/go/wiki/CodeReviewComments
+    - https://gist.github.com/lavalamp/4bd23295a9f32706a48f
+
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/coding-conventions.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/coding-conventions.md?pixel)]()

release-0.19.0/docs/devel/collab.md

+# On Collaborative Development
+
+Kubernetes is open source, but many of the people working on it do so as their day job.  In order to avoid forcing people to be "at work" effectively 24/7, we want to establish some semi-formal protocols around development.  Hopefully these rules make things go more smoothly.  If you find that this is not the case, please complain loudly.
+
+## Patches welcome
+
+First and foremost: as a potential contributor, your changes and ideas are welcome at any hour of the day or night, weekdays, weekends, and holidays.  Please do not ever hesitate to ask a question or send a PR.
+
+## Code reviews
+
+All changes must be code reviewed.  For non-maintainers this is obvious, since you can't commit anyway.  But even for maintainers, we want all changes to get at least one review, preferably (for non-trivial changes obligately) from someone who knows the areas the change touches.  For non-trivial changes we may want two reviewers.  The primary reviewer will make this decision and nominate a second reviewer, if needed.  Except for trivial changes, PRs should not be committed until relevant parties (e.g. owners of the subsystem affected by the PR) have had a reasonable chance to look at PR in their local business hours.
+
+Most PRs will find reviewers organically.  If a maintainer intends to be the primary reviewer of a PR they should set themselves as the assignee on GitHub and say so in a reply to the PR.  Only the primary reviewer of a change should actually do the merge, except in rare cases (e.g. they are unavailable in a reasonable timeframe).
+
+If a PR has gone 2 work days without an owner emerging, please poke the PR thread and ask for a reviewer to be assigned.
+
+Except for rare cases, such as trivial changes (e.g. typos, comments) or emergencies (e.g. broken builds), maintainers should not merge their own changes.
+
+Expect reviewers to request that you avoid [common go style mistakes](https://github.com/golang/go/wiki/CodeReviewComments) in your PRs.
+
+## Assigned reviews
+
+Maintainers can assign reviews to other maintainers, when appropriate. The assignee becomes the shepherd for that PR and is responsible for merging the PR once they are satisfied with it or else closing it.  The assignee might request reviews from non-maintainers.
+
+## Merge hours
+
+Maintainers will do merges of appropriately reviewed-and-approved changes during their local "business hours" (typically 7:00 am Monday to 5:00 pm (17:00h) Friday).  PRs that arrive over the weekend or on holidays will only be merged if there is a very good reason for it and if the code review requirements have been met.  Concretely this means that nobody should merge changes immediately before going to bed for the night.
+
+There may be discussion an even approvals granted outside of the above hours, but merges will generally be deferred.
+
+If a PR is considered complex or controversial, the merge of that PR should be delayed to give all interested parties in all timezones the opportunity to provide feedback.  Concretely, this means that such PRs should be held for 24 
+hours before merging.  Of course "complex" and "controversial" are left to the judgment of the people involved, but we trust that part of being a committer is the judgment required to evaluate such things honestly, and not be
+motivated by your desire (or your cube-mate's desire) to get their code merged.  Also see "Holds" below, any reviewer can issue a "hold" to indicate that the PR is in fact complicated or complex and deserves further review.
+
+PRs that are incorrectly judged to be merge-able, may be reverted and subject to re-review, if subsequent reviewers believe that they in fact are controversial or complex.
+
+
+## Holds
+
+Any maintainer or core contributor who wants to review a PR but does not have time immediately may put a hold on a PR simply by saying so on the PR discussion and offering an ETA measured in single-digit days at most.  Any PR that has a hold shall not be merged until the person who requested the hold acks the review, withdraws their hold, or is overruled by a preponderance of maintainers.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/collab.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/collab.md?pixel)]()

release-0.19.0/docs/devel/development.md

+# Development Guide
+
+# Releases and Official Builds
+
+Official releases are built in Docker containers.  Details are [here](../../build/README.md).  You can do simple builds and development with just a local Docker installation.  If want to build go locally outside of docker, please continue below.
+
+## Go development environment
+
+Kubernetes is written in [Go](http://golang.org) programming language. If you haven't set up Go development environment, please follow [this instruction](http://golang.org/doc/code.html) to install go tool and set up GOPATH. Ensure your version of Go is at least 1.3.
+
+## Clone kubernetes into GOPATH
+
+We highly recommend to put kubernetes' code into your GOPATH. For example, the following commands will download kubernetes' code under the current user's GOPATH (Assuming there's only one directory in GOPATH.):
+
+```
+$ echo $GOPATH
+/home/user/goproj
+$ mkdir -p $GOPATH/src/github.com/GoogleCloudPlatform/
+$ cd $GOPATH/src/github.com/GoogleCloudPlatform/
+$ git clone https://github.com/GoogleCloudPlatform/kubernetes.git
+```
+
+The commands above will not work if there are more than one directory in ``$GOPATH``.
+
+If you plan to do development, read about the
+[Kubernetes Github Flow](https://docs.google.com/presentation/d/1HVxKSnvlc2WJJq8b9KCYtact5ZRrzDzkWgKEfm0QO_o/pub?start=false&loop=false&delayms=3000),
+and then clone your own fork of Kubernetes as described there.
+
+## godep and dependency management
+
+Kubernetes uses [godep](https://github.com/tools/godep) to manage dependencies. It is not strictly required for building Kubernetes but it is required when managing dependencies under the Godeps/ tree, and is required by a number of the build and test scripts. Please make sure that ``godep`` is installed and in your ``$PATH``.
+
+### Installing godep
+There are many ways to build and host go binaries. Here is an easy way to get utilities like ```godep``` installed:
+
+1) Ensure that [mercurial](http://mercurial.selenic.com/wiki/Download) is installed on your system. (some of godep's dependencies use the mercurial
+source control system).  Use ```apt-get install mercurial``` or ```yum install mercurial``` on Linux, or [brew.sh](http://brew.sh) on OS X, or download
+directly from mercurial.
+
+2) Create a new GOPATH for your tools and install godep:
+```
+export GOPATH=$HOME/go-tools
+mkdir -p $GOPATH
+go get github.com/tools/godep
+```
+
+3) Add $GOPATH/bin to your path. Typically you'd add this to your ~/.profile:
+```
+export GOPATH=$HOME/go-tools
+export PATH=$PATH:$GOPATH/bin
+```
+
+### Using godep
+Here's a quick walkthrough of one way to use godeps to add or update a Kubernetes dependency into Godeps/_workspace. For more details, please see the instructions in [godep's documentation](https://github.com/tools/godep).
+
+1) Devote a directory to this endeavor:
+```
+export KPATH=$HOME/code/kubernetes
+mkdir -p $KPATH/src/github.com/GoogleCloudPlatform/kubernetes
+cd $KPATH/src/github.com/GoogleCloudPlatform/kubernetes
+git clone https://path/to/your/fork .
+# Or copy your existing local repo here. IMPORTANT: making a symlink doesn't work.
+```
+
+2) Set up your GOPATH.
+```
+# Option A: this will let your builds see packages that exist elsewhere on your system.
+export GOPATH=$KPATH:$GOPATH
+# Option B: This will *not* let your local builds see packages that exist elsewhere on your system.
+export GOPATH=$KPATH
+# Option B is recommended if you're going to mess with the dependencies.
+```
+
+3) Populate your new GOPATH.
+```
+cd $KPATH/src/github.com/GoogleCloudPlatform/kubernetes
+godep restore
+```
+
+4) Next, you can either add a new dependency or update an existing one.
+```
+# To add a new dependency, do:
+cd $KPATH/src/github.com/GoogleCloudPlatform/kubernetes
+go get path/to/dependency
+# Change code in Kubernetes to use the dependency.
+godep save ./...
+
+# To update an existing dependency, do:
+cd $KPATH/src/github.com/GoogleCloudPlatform/kubernetes
+go get -u path/to/dependency
+# Change code in Kubernetes accordingly if necessary.
+godep update path/to/dependency
+```
+
+5) Before sending your PR, it's a good idea to sanity check that your Godeps.json file is ok by re-restoring: ```godep restore```
+
+It is sometimes expedient to manually fix the /Godeps/godeps.json file to minimize the changes.
+
+Please send dependency updates in separate commits within your PR, for easier reviewing.
+
+## Hooks
+
+Before committing any changes, please link/copy these hooks into your .git
+directory. This will keep you from accidentally committing non-gofmt'd go code.
+
+```
+cd kubernetes/.git/hooks/
+ln -s ../../hooks/pre-commit .
+```
+
+## Unit tests
+
+```
+cd kubernetes
+hack/test-go.sh
+```
+
+Alternatively, you could also run:
+
+```
+cd kubernetes
+godep go test ./...
+```
+
+If you only want to run unit tests in one package, you could run ``godep go test`` under the package directory. For example, the following commands will run all unit tests in package kubelet:
+
+```
+$ cd kubernetes # step into kubernetes' directory.
+$ cd pkg/kubelet
+$ godep go test
+# some output from unit tests
+PASS
+ok      github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet   0.317s
+```
+
+## Coverage
+
+Currently, collecting coverage is only supported for the Go unit tests.
+
+To run all unit tests and generate an HTML coverage report, run the following:
+
+```
+cd kubernetes
+KUBE_COVER=y hack/test-go.sh
+```
+
+At the end of the run, an the HTML report will be generated with the path printed to stdout.
+
+To run tests and collect coverage in only one package, pass its relative path under the `kubernetes` directory as an argument, for example:
+```
+cd kubernetes
+KUBE_COVER=y hack/test-go.sh pkg/kubectl
+```
+
+Multiple arguments can be passed, in which case the coverage results will be combined for all tests run.
+
+Coverage results for the project can also be viewed on [Coveralls](https://coveralls.io/r/GoogleCloudPlatform/kubernetes), and are continuously updated as commits are merged. Additionally, all pull requests which spawn a Travis build will report unit test coverage results to Coveralls.
+
+## Integration tests
+
+You need an [etcd](https://github.com/coreos/etcd/releases/tag/v2.0.0) in your path, please make sure it is installed and in your ``$PATH``.
+```
+cd kubernetes
+hack/test-integration.sh
+```
+
+## End-to-End tests
+
+You can run an end-to-end test which will bring up a master and two minions, perform some tests, and then tear everything down. Make sure you have followed the getting started steps for your chosen cloud platform (which might involve changing the `KUBERNETES_PROVIDER` environment variable to something other than "gce".
+```
+cd kubernetes
+hack/e2e-test.sh
+```
+
+Pressing control-C should result in an orderly shutdown but if something goes wrong and you still have some VMs running you can force a cleanup with this command:
+```
+go run hack/e2e.go --down
+```
+
+### Flag options
+See the flag definitions in `hack/e2e.go` for more options, such as reusing an existing cluster, here is an overview:
+
+```sh
+# Build binaries for testing
+go run hack/e2e.go --build
+
+# Create a fresh cluster.  Deletes a cluster first, if it exists
+go run hack/e2e.go --up
+
+# Create a fresh cluster at a specific release version.
+go run hack/e2e.go --up --version=0.7.0
+
+# Test if a cluster is up.
+go run hack/e2e.go --isup
+
+# Push code to an existing cluster
+go run hack/e2e.go --push
+
+# Push to an existing cluster, or bring up a cluster if it's down.
+go run hack/e2e.go --pushup
+
+# Run all tests
+go run hack/e2e.go --test
+
+# Run tests matching the regex "Pods.*env"
+go run hack/e2e.go -v -test --test_args="--ginkgo.focus=Pods.*env"
+
+# Alternately, if you have the e2e cluster up and no desire to see the event stream, you can run ginkgo-e2e.sh directly:
+hack/ginkgo-e2e.sh --ginkgo.focus=Pods.*env
+```
+
+### Combining flags
+```sh
+# Flags can be combined, and their actions will take place in this order:
+# -build, -push|-up|-pushup, -test|-tests=..., -down
+# e.g.:
+go run hack/e2e.go -build -pushup -test -down
+
+# -v (verbose) can be added if you want streaming output instead of only
+# seeing the output of failed commands.
+
+# -ctl can be used to quickly call kubectl against your e2e cluster. Useful for
+# cleaning up after a failed test or viewing logs. Use -v to avoid suppressing
+# kubectl output.
+go run hack/e2e.go -v -ctl='get events'
+go run hack/e2e.go -v -ctl='delete pod foobar'
+```
+
+## Conformance testing
+End-to-end testing, as described above, is for [development
+distributions](../../docs/devel/writing-a-getting-started-guide.md).  A conformance test is used on
+a [versioned distro](../../docs/devel/writing-a-getting-started-guide.md).
+
+The conformance test runs a subset of the e2e-tests against a manually-created cluster.  It does not
+require support for up/push/down and other operations.  To run a conformance test, you need to know the
+IP of the master for your cluster and the authorization arguments to use.  The conformance test is
+intended to run against a cluster at a specific binary release of Kubernetes.
+See [conformance-test.sh](../../hack/conformance-test.sh).
+
+## Testing out flaky tests
+[Instructions here](flaky-tests.md)
+
+## Keeping your development fork in sync
+
+One time after cloning your forked repo:
+
+```
+git remote add upstream https://github.com/GoogleCloudPlatform/kubernetes.git
+```
+
+Then each time you want to sync to upstream:
+
+```
+git fetch upstream
+git rebase upstream/master
+```
+
+If you have write access to the main repository, you should modify your git configuration so that
+you can't accidentally push to upstream:
+
+```
+git remote set-url --push upstream no_push
+```
+
+## Regenerating the CLI documentation
+
+```
+hack/run-gendocs.sh
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/development.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/development.md?pixel)]()

release-0.19.0/docs/devel/faster_reviews.md

+# How to get faster PR reviews
+
+Most of what is written here is not at all specific to Kubernetes, but it bears
+being written down in the hope that it will occasionally remind people of "best
+practices" around code reviews.
+
+You've just had a brilliant idea on how to make Kubernetes better.  Let's call
+that idea "FeatureX".  Feature X is not even that complicated.  You have a
+pretty good idea of how to implement it.  You jump in and implement it, fixing a
+bunch of stuff along the way.  You send your PR - this is awesome!  And it sits.
+And sits.  A week goes by and nobody reviews it.  Finally someone offers a few
+comments, which you fix up and wait for more review.  And you wait.  Another
+week or two goes by.  This is horrible.
+
+What went wrong?  One particular problem that comes up frequently is this - your
+PR is too big to review.  You've touched 39 files and have 8657 insertions.
+When your would-be reviewers pull up the diffs they run away - this PR is going
+to take 4 hours to review and they don't have 4 hours right now.  They'll get to it
+later, just as soon as they have more free time (ha!).
+
+Let's talk about how to avoid this.
+
+## 1. Don't build a cathedral in one PR
+
+Are you sure FeatureX is something the Kubernetes team wants or will accept, or
+that it is implemented to fit with other changes in flight?  Are you willing to
+bet a few days or weeks of work on it?  If you have any doubt at all about the
+usefulness of your feature or the design - make a proposal doc or a sketch PR
+or both.  Write or code up just enough to express the idea and the design and
+why you made those choices, then get feedback on this.  Now, when we ask you to
+change a bunch of facets of the design, you don't have to re-write it all.
+
+## 2. Smaller diffs are exponentially better
+
+Small PRs get reviewed faster and are more likely to be correct than big ones.
+Let's face it - attention wanes over time.  If your PR takes 60 minutes to
+review, I almost guarantee that the reviewer's eye for details is not as keen in
+the last 30 minutes as it was in the first.  This leads to multiple rounds of
+review when one might have sufficed.  In some cases the review is delayed in its
+entirety by the need for a large contiguous block of time to sit and read your
+code.
+
+Whenever possible, break up your PRs into multiple commits.  Making a series of
+discrete commits is a powerful way to express the evolution of an idea or the
+different ideas that make up a single feature.  There's a balance to be struck,
+obviously.  If your commits are too small they become more cumbersome to deal
+with.  Strive to group logically distinct ideas into commits.
+
+For example, if you found that FeatureX needed some "prefactoring" to fit in,
+make a commit that JUST does that prefactoring.  Then make a new commit for
+FeatureX.  Don't lump unrelated things together just because you didn't think
+about prefactoring.  If you need to, fork a new branch, do the prefactoring
+there and send a PR for that.  If you can explain why you are doing seemingly
+no-op work ("it makes the FeatureX change easier, I promise") we'll probably be
+OK with it.
+
+Obviously, a PR with 25 commits is still very cumbersome to review, so use
+common sense.
+
+## 3. Multiple small PRs are often better than multiple commits
+
+If you can extract whole ideas from your PR and send those as PRs of their own,
+you can avoid the painful problem of continually rebasing.  Kubernetes is a
+fast-moving codebase - lock in your changes ASAP, and make merges be someone
+else's problem.
+
+Obviously, we want every PR to be useful on its own, so you'll have to use
+common sense in deciding what can be a PR vs what should be a commit in a larger
+PR.  Rule of thumb - if this commit or set of commits is directly related to
+FeatureX and nothing else, it should probably be part of the FeatureX PR.  If
+you can plausibly imagine someone finding value in this commit outside of
+FeatureX, try it as a PR.
+
+Don't worry about flooding us with PRs.  We'd rather have 100 small, obvious PRs
+than 10 unreviewable monoliths.
+
+## 4. Don't rename, reformat, comment, etc in the same PR
+
+Often, as you are implementing FeatureX, you find things that are just wrong.
+Bad comments, poorly named functions, bad structure, weak type-safety.  You
+should absolutely fix those things (or at least file issues, please) - but not
+in this PR.  See the above points - break unrelated changes out into different
+PRs or commits.  Otherwise your diff will have WAY too many changes, and your
+reviewer won't see the forest because of all the trees.
+
+## 5. Comments matter
+
+Read up on GoDoc - follow those general rules.  If you're writing code and you
+think there is any possible chance that someone might not understand why you did
+something (or that you won't remember what you yourself did), comment it.  If
+you think there's something pretty obvious that we could follow up on, add a
+TODO.  Many code-review comments are about this exact issue.
+
+## 5. Tests are almost always required
+
+Nothing is more frustrating than doing a review, only to find that the tests are
+inadequate or even entirely absent.  Very few PRs can touch code and NOT touch
+tests.  If you don't know how to test FeatureX - ask!  We'll be happy to help
+you design things for easy testing or to suggest appropriate test cases.
+
+## 6. Look for opportunities to generify
+
+If you find yourself writing something that touches a lot of modules, think hard
+about the dependencies you are introducing between packages.  Can some of what
+you're doing be made more generic and moved up and out of the FeatureX package?
+Do you need to use a function or type from an otherwise unrelated package?  If
+so, promote!  We have places specifically for hosting more generic code.
+
+Likewise if FeatureX is similar in form to FeatureW which was checked in last
+month and it happens to exactly duplicate some tricky stuff from FeatureW,
+consider prefactoring core logic out and using it in both FeatureW and FeatureX.
+But do that in a different commit or PR, please.
+
+## 7. Fix feedback in a new commit
+
+Your reviewer has finally sent you some feedback on FeatureX.  You make a bunch
+of changes and ... what?  You could patch those into your commits with git
+"squash" or "fixup" logic.  But that makes your changes hard to verify.  Unless
+your whole PR is pretty trivial, you should instead put your fixups into a new
+commit and re-push.  Your reviewer can then look at that commit on its own - so
+much faster to review than starting over.
+
+We might still ask you to clean up your commits at the very end, for the sake
+of a more readable history.
+
+## 8. KISS, YAGNI, MVP, etc
+
+Sometimes we need to remind each other of core tenets of software design - Keep
+It Simple, You Aren't Gonna Need It, Minimum Viable Product, and so on.  Adding
+features "because we might need it later" is antithetical to software that
+ships.  Add the things you need NOW and (ideally) leave room for things you
+might need later - but don't implement them now.
+
+## 9. Push back
+
+We understand that it is hard to imagine, but sometimes we make mistakes.  It's
+OK to push back on changes requested during a review.  If you have a good reason
+for doing something a certain way, you are absolutely allowed to debate the
+merits of a requested change.  You might be overruled, but you might also
+prevail.  We're mostly pretty reasonable people.  Mostly.
+
+## 10. I'm still getting stalled - help?!
+
+So, you've done all that and you still aren't getting any PR love?  Here's some
+things you can do that might help kick a stalled process along:
+
+   * Make sure that your PR has an assigned reviewer (assignee in GitHub).  If
+     this is not the case, reply to the PR comment stream asking for one to be
+     assigned.
+
+   * Ping the assignee (@username) on the PR comment stream asking for an
+     estimate of when they can get to it.
+
+   * Ping the assignee by email (many of us have email addresses that are well
+     published or are the same as our GitHub handle @google.com or @redhat.com).
+
+If you think you have fixed all the issues in a round of review, and you haven't
+heard back, you should ping the reviewer (assignee) on the comment stream with a
+"please take another look" (PTAL) or similar comment indicating you are done and
+you think it is ready for re-review.  In fact, this is probably a good habit for
+all PRs.
+
+One phenomenon of open-source projects (where anyone can comment on any issue)
+is the dog-pile - your PR gets so many comments from so many people it becomes
+hard to follow.  In this situation you can ask the primary reviewer
+(assignee) whether they want you to fork a new PR to clear out all the comments.
+Remember: you don't HAVE to fix every issue raised by every person who feels
+like commenting, but you should at least answer reasonable comments with an
+explanation.
+
+## Final: Use common sense
+
+Obviously, none of these points are hard rules.  There is no document that can
+take the place of common sense and good taste.  Use your best judgment, but put
+a bit of thought into how your work can be made easier to review.  If you do
+these things your PRs will flow much more easily.
+
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/faster_reviews.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/faster_reviews.md?pixel)]()

release-0.19.0/docs/devel/flaky-tests.md

+# Hunting flaky tests in Kubernetes
+Sometimes unit tests are flaky.  This means that due to (usually) race conditions, they will occasionally fail, even though most of the time they pass.
+
+We have a goal of 99.9% flake free tests.  This means that there is only one flake in one thousand runs of a test.
+
+Running a test 1000 times on your own machine can be tedious and time consuming.  Fortunately, there is a better way to achieve this using Kubernetes.
+
+_Note: these instructions are mildly hacky for now, as we get run once semantics and logging they will get better_
+
+There is a testing image ```brendanburns/flake``` up on the docker hub.  We will use this image to test our fix.
+
+Create a replication controller with the following config:
+```yaml
+apiVersion: v1
+kind: ReplicationController
+metadata:
+  name: flakecontroller
+spec:
+  replicas: 24
+  template:
+    metadata:
+      labels:
+        name: flake
+    spec:
+      containers:
+      - name: flake
+        image: brendanburns/flake
+        env:
+        - name: TEST_PACKAGE
+          value: pkg/tools
+        - name: REPO_SPEC
+          value: https://github.com/GoogleCloudPlatform/kubernetes
+```
+Note that we omit the labels and the selector fields of the replication controller, because they will be populated from the labels field of the pod template by default.
+
+```
+kubectl create -f controller.yaml
+```
+
+This will spin up 24 instances of the test.  They will run to completion, then exit, and the kubelet will restart them, accumulating more and more runs of the test.
+You can examine the recent runs of the test by calling ```docker ps -a``` and looking for tasks that exited with non-zero exit codes. Unfortunately, docker ps -a only keeps around the exit status of the last 15-20 containers with the same image, so you have to check them frequently.
+You can use this script to automate checking for failures, assuming your cluster is running on GCE and has four nodes:
+
+```sh
+echo "" > output.txt
+for i in {1..4}; do
+  echo "Checking kubernetes-minion-${i}"
+  echo "kubernetes-minion-${i}:" >> output.txt
+  gcloud compute ssh "kubernetes-minion-${i}" --command="sudo docker ps -a" >> output.txt
+done
+grep "Exited ([^0])" output.txt
+```
+
+Eventually you will have sufficient runs for your purposes. At that point you can stop and delete the replication controller by running:
+
+```sh
+kubectl stop replicationcontroller flakecontroller
+```
+
+If you do a final check for flakes with ```docker ps -a```, ignore tasks that exited -1, since that's what happens when you stop the replication controller.
+
+Happy flake hunting!
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/flaky-tests.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/flaky-tests.md?pixel)]()

release-0.19.0/docs/devel/issues.md

+GitHub Issues for the Kubernetes Project
+========================================
+
+A list quick overview of how we will review and prioritize incoming issues at https://github.com/GoogleCloudPlatform/kubernetes/issues
+
+Priorities
+----------
+
+We will use GitHub issue labels for prioritization.  The absence of a priority label means the bug has not been reviewed and prioritized yet.
+
+Definitions
+-----------
+* P0 - something broken for users, build broken, or critical security issue.  Someone must drop everything and work on it.
+* P1 - must fix for earliest possible binary release (every two weeks)
+* P2 - should be fixed in next major release version
+* P3 - default priority for lower importance bugs that we still want to track and plan to fix at some point
+* design - priority/design is for issues that are used to track design discussions
+* support - priority/support is used for issues tracking user support requests
+* untriaged - anything without a priority/X label will be considered untriaged
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/issues.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/issues.md?pixel)]()

release-0.19.0/docs/devel/logging.md

+Logging Conventions
+===================
+
+The following conventions for the glog levels to use.  [glog](http://godoc.org/github.com/golang/glog) is globally preferred to [log](http://golang.org/pkg/log/) for better runtime control.
+
+* glog.Errorf() - Always an error
+* glog.Warningf() - Something unexpected, but probably not an error
+* glog.Infof() has multiple levels:
+  * glog.V(0) - Generally useful for this to ALWAYS be visible to an operator
+    * Programmer errors
+    * Logging extra info about a panic
+    * CLI argument handling
+  * glog.V(1) - A reasonable default log level if you don't want verbosity.
+    * Information about config (listening on X, watching Y)
+    * Errors that repeat frequently that relate to conditions that can be corrected (pod detected as unhealthy)
+  * glog.V(2) - Useful steady state information about the service and important log messages that may correlate to significant changes in the system.  This is the recommended default log level for most systems.
+    * Logging HTTP requests and their exit code
+    * System state changing (killing pod)
+    * Controller state change events (starting pods)
+    * Scheduler log messages
+  * glog.V(3) - Extended information about changes
+    * More info about system state changes
+  * glog.V(4) - Debug level verbosity (for now)
+    * Logging in particularly thorny parts of code where you may want to come back later and check it
+
+As per the comments, the practical default level is V(2).  Developers and QE environments may wish to run at V(3) or V(4). If you wish to change the log level, you can pass in `-v=X` where X is the desired maximum level to log.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/logging.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/logging.md?pixel)]()

release-0.19.0/docs/devel/profiling.md

+# Profiling Kubernetes
+
+This document explain how to plug in profiler and how to profile Kubernetes services.
+
+## Profiling library
+
+Go comes with inbuilt 'net/http/pprof' profiling library and profiling web service. The way service works is binding debug/pprof/ subtree on a running webserver to the profiler. Reading from subpages of debug/pprof returns pprof-formatted profiles of the running binary. The output can be processed offline by the tool of choice, or used as an input to handy 'go tool pprof', which can graphically represent the result.
+
+## Adding profiling to services to APIserver.
+
+TL;DR: Add lines:
+```
+  m.mux.HandleFunc("/debug/pprof/", pprof.Index)
+  m.mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
+  m.mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
+```
+to the init(c *Config) method in 'pkg/master/master.go' and import 'net/http/pprof' package.
+
+In most use cases to use profiler service it's enough to do 'import _ net/http/pprof', which automatically registers a handler in the default http.Server. Slight inconvenience is that APIserver uses default server for intra-cluster communication, so plugging profiler to it is not really useful. In 'pkg/master/server/server.go' more servers are created and started as separate goroutines. The one that is usually serving external traffic is secureServer. The handler for this traffic is defined in 'pkg/master/master.go' and stored in Handler variable. It is created from HTTP multiplexer, so the only thing that needs to be done is adding profiler handler functions to this multiplexer. This is exactly what lines after TL;DR do.
+
+## Connecting to the profiler
+Even when running profiler I found not really straightforward to use 'go tool pprof' with it. The problem is that at least for dev purposes certificates generated for APIserver are not signed by anyone trusted and because secureServer serves only secure traffic it isn't straightforward to connect to the service. The best workaround I found is by creating an ssh tunnel from the kubernetes_master open unsecured port to some external server, and use this server as a proxy. To save everyone looking for correct ssh flags, it is done by running:
+```
+  ssh kubernetes_master -L<local_port>:localhost:8080
+```
+or analogous one for you Cloud provider. Afterwards you can e.g. run
+```
+go tool pprof http://localhost:<local_port>/debug/pprof/profile
+```
+to get 30 sec. CPU profile.
+
+## Contention profiling
+
+To enable contention profiling you need to add line ```rt.SetBlockProfileRate(1)``` in addition to ```m.mux.HandleFunc(...)``` added before (```rt``` stands for ```runtime``` in ```master.go```). This enables 'debug/pprof/block' subpage, which can be used as an input to ```go tool pprof```.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/profiling.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/profiling.md?pixel)]()

release-0.19.0/docs/devel/pull-requests.md

+Pull Request Process
+====================
+
+An overview of how we will manage old or out-of-date pull requests.
+
+Process
+-------
+
+We will close any pull requests older than two weeks.
+
+Exceptions can be made for PRs that have active review comments, or that are awaiting other dependent PRs.  Closed pull requests are easy to recreate, and little work is lost by closing a pull request that subsequently needs to be reopened.
+
+We want to limit the total number of PRs in flight to:
+* Maintain a clean project
+* Remove old PRs that would be difficult to rebase as the underlying code has changed over time
+* Encourage code velocity
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/pull-requests.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/pull-requests.md?pixel)]()

release-0.19.0/docs/devel/releasing.dot

+// Build it with:
+//   $ dot -Tsvg releasing.dot >releasing.svg
+
+digraph tagged_release {
+  size = "5,5"
+  // Arrows go up.
+  rankdir = BT
+  subgraph left {
+    // Group the left nodes together.
+    ci012abc -> pr101 -> ci345cde -> pr102
+    style = invis
+  }
+  subgraph right {
+    // Group the right nodes together.
+    version_commit -> dev_commit
+    style = invis
+  }
+  { // Align the version commit and the info about it.
+    rank = same
+    // Align them with pr101
+    pr101
+    version_commit
+    // release_info shows the change in the commit.
+    release_info
+  }
+  { // Align the dev commit and the info about it.
+    rank = same
+    // Align them with 345cde
+    ci345cde
+    dev_commit
+    dev_info
+  }
+  // Join the nodes from subgraph left.
+  pr99 -> ci012abc
+  pr102 -> pr100
+  // Do the version node.
+  pr99 -> version_commit
+  dev_commit -> pr100
+  tag -> version_commit
+  pr99 [
+    label = "Merge PR #99"
+    shape = box
+    fillcolor = "#ccccff"
+    style = "filled"
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+  ci012abc [
+    label = "012abc"
+    shape = circle
+    fillcolor = "#ffffcc"
+    style = "filled"
+    fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
+  ];
+  pr101 [
+    label = "Merge PR #101"
+    shape = box
+    fillcolor = "#ccccff"
+    style = "filled"
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+  ci345cde [
+    label = "345cde"
+    shape = circle
+    fillcolor = "#ffffcc"
+    style = "filled"
+    fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
+  ];
+  pr102 [
+    label = "Merge PR #102"
+    shape = box
+    fillcolor = "#ccccff"
+    style = "filled"
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+  version_commit [
+    label = "678fed"
+    shape = circle
+    fillcolor = "#ccffcc"
+    style = "filled"
+    fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
+  ];
+  dev_commit [
+    label = "456dcb"
+    shape = circle
+    fillcolor = "#ffffcc"
+    style = "filled"
+    fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
+  ];
+  pr100 [
+    label = "Merge PR #100"
+    shape = box
+    fillcolor = "#ccccff"
+    style = "filled"
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+  release_info [
+    label = "pkg/version/base.go:\ngitVersion = \"v0.5\";"
+    shape = none
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+  dev_info [
+    label = "pkg/version/base.go:\ngitVersion = \"v0.5-dev\";"
+    shape = none
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+  tag [
+    label = "$ git tag -a v0.5"
+    fillcolor = "#ffcccc"
+    style = "filled"
+    fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
+  ];
+}
+

release-0.19.0/docs/devel/releasing.md

+# Releasing Kubernetes
+
+This document explains how to create a Kubernetes release (as in version) and
+how the version information gets embedded into the built binaries.
+
+## Origin of the Sources
+
+Kubernetes may be built from either a git tree (using `hack/build-go.sh`) or
+from a tarball (using either `hack/build-go.sh` or `go install`) or directly by
+the Go native build system (using `go get`).
+
+When building from git, we want to be able to insert specific information about
+the build tree at build time. In particular, we want to use the output of `git
+describe` to generate the version of Kubernetes and the status of the build
+tree (add a `-dirty` prefix if the tree was modified.)
+
+When building from a tarball or using the Go build system, we will not have
+access to the information about the git tree, but we still want to be able to
+tell whether this build corresponds to an exact release (e.g. v0.3) or is
+between releases (e.g. at some point in development between v0.3 and v0.4).
+
+## Version Number Format
+
+In order to account for these use cases, there are some specific formats that
+may end up representing the Kubernetes version. Here are a few examples:
+
+- **v0.5**: This is official version 0.5 and this version will only be used
+  when building from a clean git tree at the v0.5 git tag, or from a tree
+  extracted from the tarball corresponding to that specific release.
+- **v0.5-15-g0123abcd4567**: This is the `git describe` output and it indicates
+  that we are 15 commits past the v0.5 release and that the SHA1 of the commit
+  where the binaries were built was `0123abcd4567`. It is only possible to have
+  this level of detail in the version information when building from git, not
+  when building from a tarball.
+- **v0.5-15-g0123abcd4567-dirty** or **v0.5-dirty**: The extra `-dirty` prefix
+  means that the tree had local modifications or untracked files at the time of
+  the build, so there's no guarantee that the source code matches exactly the
+  state of the tree at the `0123abcd4567` commit or at the `v0.5` git tag
+  (resp.)
+- **v0.5-dev**: This means we are building from a tarball or using `go get` or,
+  if we have a git tree, we are using `go install` directly, so it is not
+  possible to inject the git version into the build information. Additionally,
+  this is not an official release, so the `-dev` prefix indicates that the
+  version we are building is after `v0.5` but before `v0.6`. (There is actually
+  an exception where a commit with `v0.5-dev` is not present on `v0.6`, see
+  later for details.)
+
+## Injecting Version into Binaries
+
+In order to cover the different build cases, we start by providing information
+that can be used when using only Go build tools or when we do not have the git
+version information available.
+
+To be able to provide a meaningful version in those cases, we set the contents
+of variables in a Go source file that will be used when no overrides are
+present.
+
+We are using `pkg/version/base.go` as the source of versioning in absence of
+information from git. Here is a sample of that file's contents:
+
+```
+  var (
+      gitVersion   string = "v0.4-dev"  // version from git, output of $(git describe)
+      gitCommit    string = ""          // sha1 from git, output of $(git rev-parse HEAD)
+  )
+```
+
+This means a build with `go install` or `go get` or a build from a tarball will
+yield binaries that will identify themselves as `v0.4-dev` and will not be able
+to provide you with a SHA1.
+
+To add the extra versioning information when building from git, the
+`hack/build-go.sh` script will gather that information (using `git describe` and
+`git rev-parse`) and then create a `-ldflags` string to pass to `go install` and
+tell the Go linker to override the contents of those variables at build time. It
+can, for instance, tell it to override `gitVersion` and set it to
+`v0.4-13-g4567bcdef6789-dirty` and set `gitCommit` to `4567bcdef6789...` which
+is the complete SHA1 of the (dirty) tree used at build time.
+
+## Handling Official Versions
+
+Handling official versions from git is easy, as long as there is an annotated
+git tag pointing to a specific version then `git describe` will return that tag
+exactly which will match the idea of an official version (e.g. `v0.5`).
+
+Handling it on tarballs is a bit harder since the exact version string must be
+present in `pkg/version/base.go` for it to get embedded into the binaries. But
+simply creating a commit with `v0.5` on its own would mean that the commits
+coming after it would also get the `v0.5` version when built from tarball or `go
+get` while in fact they do not match `v0.5` (the one that was tagged) exactly.
+
+To handle that case, creating a new release should involve creating two adjacent
+commits where the first of them will set the version to `v0.5` and the second
+will set it to `v0.5-dev`. In that case, even in the presence of merges, there
+will be a single commit where the exact `v0.5` version will be used and all
+others around it will either have `v0.4-dev` or `v0.5-dev`.
+
+The diagram below illustrates it.
+
+![Diagram of git commits involved in the release](./releasing.png)
+
+After working on `v0.4-dev` and merging PR 99 we decide it is time to release
+`v0.5`. So we start a new branch, create one commit to update
+`pkg/version/base.go` to include `gitVersion = "v0.5"` and `git commit` it.
+
+We test it and make sure everything is working as expected.
+
+Before sending a PR for it, we create a second commit on that same branch,
+updating `pkg/version/base.go` to include `gitVersion = "v0.5-dev"`. That will
+ensure that further builds (from tarball or `go install`) on that tree will
+always include the `-dev` prefix and will not have a `v0.5` version (since they
+do not match the official `v0.5` exactly.)
+
+We then send PR 100 with both commits in it.
+
+Once the PR is accepted, we can use `git tag -a` to create an annotated tag
+*pointing to the one commit* that has `v0.5` in `pkg/version/base.go` and push
+it to GitHub. (Unfortunately GitHub tags/releases are not annotated tags, so
+this needs to be done from a git client and pushed to GitHub using SSH.)
+
+## Parallel Commits
+
+While we are working on releasing `v0.5`, other development takes place and
+other PRs get merged. For instance, in the example above, PRs 101 and 102 get
+merged to the master branch before the versioning PR gets merged.
+
+This is not a problem, it is only slightly inaccurate that checking out the tree
+at commit `012abc` or commit `345cde` or at the commit of the merges of PR 101
+or 102 will yield a version of `v0.4-dev` *but* those commits are not present in
+`v0.5`.
+
+In that sense, there is a small window in which commits will get a
+`v0.4-dev` or `v0.4-N-gXXX` label and while they're indeed later than `v0.4`
+but they are not really before `v0.5` in that `v0.5` does not contain those
+commits.
+
+Unfortunately, there is not much we can do about it. On the other hand, other
+projects seem to live with that and it does not really become a large problem.
+
+As an example, Docker commit a327d9b91edf has a `v1.1.1-N-gXXX` label but it is
+not present in Docker `v1.2.0`:
+
+```
+  $ git describe a327d9b91edf
+  v1.1.1-822-ga327d9b91edf
+
+  $ git log --oneline v1.2.0..a327d9b91edf
+  a327d9b91edf Fix data space reporting from Kb/Mb to KB/MB
+
+  (Non-empty output here means the commit is not present on v1.2.0.)
+```
+
+## Release Notes
+
+No official release should be made final without properly matching release notes.
+
+There should be made available, per release, a small summary, preamble, of the
+major changes, both in terms of feature improvements/bug fixes and notes about
+functional feature changes (if any) regarding the previous released version so
+that the BOM regarding updating to it gets as obvious and trouble free as possible.
+
+After this summary, preamble, all the relevant PRs/issues that got in that
+version should be listed and linked together with a small summary understandable
+by plain mortals (in a perfect world PR/issue's title would be enough but often
+it is just too cryptic/geeky/domain-specific that it isn't).
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/releasing.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/releasing.md?pixel)]()

release-0.19.0/docs/devel/releasing.svg

+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+ "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by graphviz version 2.36.0 (20140111.2315)
+ -->
+<!-- Title: tagged_release Pages: 1 -->
+<svg width="257pt" height="360pt"
+ viewBox="0.00 0.00 257.33 360.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+<g id="graph0" class="graph" transform="scale(0.649819 0.649819) rotate(0) translate(4 550)">
+<title>tagged_release</title>
+<polygon fill="white" stroke="none" points="-4,4 -4,-550 392,-550 392,4 -4,4"/>
+<!-- ci012abc -->
+<g id="node1" class="node"><title>ci012abc</title>
+<ellipse fill="#ffffcc" stroke="black" cx="56" cy="-115" rx="42.7926" ry="42.7926"/>
+<text text-anchor="middle" x="56" y="-111.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">012abc</text>
+</g>
+<!-- pr101 -->
+<g id="node2" class="node"><title>pr101</title>
+<polygon fill="#ccccff" stroke="black" points="112,-255 0,-255 0,-219 112,-219 112,-255"/>
+<text text-anchor="middle" x="56" y="-233.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #101</text>
+</g>
+<!-- ci012abc&#45;&gt;pr101 -->
+<g id="edge1" class="edge"><title>ci012abc&#45;&gt;pr101</title>
+<path fill="none" stroke="black" d="M56,-157.97C56,-174.83 56,-193.784 56,-208.789"/>
+<polygon fill="black" stroke="black" points="52.5001,-208.93 56,-218.93 59.5001,-208.93 52.5001,-208.93"/>
+</g>
+<!-- ci345cde -->
+<g id="node3" class="node"><title>ci345cde</title>
+<ellipse fill="#ffffcc" stroke="black" cx="62" cy="-359" rx="42.7926" ry="42.7926"/>
+<text text-anchor="middle" x="62" y="-355.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">345cde</text>
+</g>
+<!-- pr101&#45;&gt;ci345cde -->
+<g id="edge2" class="edge"><title>pr101&#45;&gt;ci345cde</title>
+<path fill="none" stroke="black" d="M56.8597,-255.193C57.5237,-268.473 58.4796,-287.592 59.3874,-305.748"/>
+<polygon fill="black" stroke="black" points="55.904,-306.17 59.8991,-315.982 62.8953,-305.82 55.904,-306.17"/>
+</g>
+<!-- pr102 -->
+<g id="node4" class="node"><title>pr102</title>
+<polygon fill="#ccccff" stroke="black" points="129,-474 17,-474 17,-438 129,-438 129,-474"/>
+<text text-anchor="middle" x="73" y="-452.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #102</text>
+</g>
+<!-- ci345cde&#45;&gt;pr102 -->
+<g id="edge3" class="edge"><title>ci345cde&#45;&gt;pr102</title>
+<path fill="none" stroke="black" d="M66.8248,-401.668C67.8523,-410.542 68.9117,-419.692 69.8567,-427.853"/>
+<polygon fill="black" stroke="black" points="66.3936,-428.375 71.0207,-437.906 73.3472,-427.57 66.3936,-428.375"/>
+</g>
+<!-- pr100 -->
+<g id="node10" class="node"><title>pr100</title>
+<polygon fill="#ccccff" stroke="black" points="174,-546 62,-546 62,-510 174,-510 174,-546"/>
+<text text-anchor="middle" x="118" y="-524.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #100</text>
+</g>
+<!-- pr102&#45;&gt;pr100 -->
+<g id="edge6" class="edge"><title>pr102&#45;&gt;pr100</title>
+<path fill="none" stroke="black" d="M84.1236,-474.303C89.355,-482.441 95.6999,-492.311 101.478,-501.299"/>
+<polygon fill="black" stroke="black" points="98.6526,-503.377 107.004,-509.896 104.541,-499.591 98.6526,-503.377"/>
+</g>
+<!-- version_commit -->
+<g id="node5" class="node"><title>version_commit</title>
+<ellipse fill="#ccffcc" stroke="black" cx="173" cy="-237" rx="42.7926" ry="42.7926"/>
+<text text-anchor="middle" x="173" y="-233.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">678fed</text>
+</g>
+<!-- dev_commit -->
+<g id="node6" class="node"><title>dev_commit</title>
+<ellipse fill="#ffffcc" stroke="black" cx="169" cy="-359" rx="42.7926" ry="42.7926"/>
+<text text-anchor="middle" x="169" y="-355.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">456dcb</text>
+</g>
+<!-- version_commit&#45;&gt;dev_commit -->
+<g id="edge4" class="edge"><title>version_commit&#45;&gt;dev_commit</title>
+<path fill="none" stroke="black" d="M171.601,-279.97C171.322,-288.326 171.027,-297.195 170.739,-305.844"/>
+<polygon fill="black" stroke="black" points="167.24,-305.74 170.405,-315.851 174.236,-305.973 167.24,-305.74"/>
+</g>
+<!-- dev_commit&#45;&gt;pr100 -->
+<g id="edge8" class="edge"><title>dev_commit&#45;&gt;pr100</title>
+<path fill="none" stroke="black" d="M158.36,-400.568C152.438,-422.433 144.719,-449.815 137,-474 134.253,-482.606 131.013,-491.906 127.994,-500.278"/>
+<polygon fill="black" stroke="black" points="124.616,-499.325 124.47,-509.919 131.191,-501.729 124.616,-499.325"/>
+</g>
+<!-- release_info -->
+<g id="node7" class="node"><title>release_info</title>
+<text text-anchor="middle" x="304" y="-240.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">pkg/version/base.go:</text>
+<text text-anchor="middle" x="304" y="-225.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">gitVersion = &quot;v0.5&quot;;</text>
+</g>
+<!-- dev_info -->
+<g id="node8" class="node"><title>dev_info</title>
+<text text-anchor="middle" x="309" y="-362.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">pkg/version/base.go:</text>
+<text text-anchor="middle" x="309" y="-347.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">gitVersion = &quot;v0.5&#45;dev&quot;;</text>
+</g>
+<!-- pr99 -->
+<g id="node9" class="node"><title>pr99</title>
+<polygon fill="#ccccff" stroke="black" points="143,-36 39,-36 39,-0 143,-0 143,-36"/>
+<text text-anchor="middle" x="91" y="-14.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #99</text>
+</g>
+<!-- pr99&#45;&gt;ci012abc -->
+<g id="edge5" class="edge"><title>pr99&#45;&gt;ci012abc</title>
+<path fill="none" stroke="black" d="M84.5805,-36.4245C81.5586,-44.6267 77.7879,-54.8615 73.9865,-65.1795"/>
+<polygon fill="black" stroke="black" points="70.6804,-64.0292 70.5075,-74.6226 77.2488,-66.4492 70.6804,-64.0292"/>
+</g>
+<!-- pr99&#45;&gt;version_commit -->
+<g id="edge7" class="edge"><title>pr99&#45;&gt;version_commit</title>
+<path fill="none" stroke="black" d="M97.4344,-36.0276C109.585,-68.1826 136.317,-138.924 154.498,-187.038"/>
+<polygon fill="black" stroke="black" points="151.318,-188.523 158.127,-196.64 157.866,-186.048 151.318,-188.523"/>
+</g>
+<!-- tag -->
+<g id="node11" class="node"><title>tag</title>
+<ellipse fill="#ffcccc" stroke="black" cx="226" cy="-115" rx="71.4873" ry="18"/>
+<text text-anchor="middle" x="226" y="-111.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">$ git tag &#45;a v0.5</text>
+</g>
+<!-- tag&#45;&gt;version_commit -->
+<g id="edge9" class="edge"><title>tag&#45;&gt;version_commit</title>
+<path fill="none" stroke="black" d="M218.519,-132.939C212.168,-147.318 202.736,-168.673 194.103,-188.22"/>
+<polygon fill="black" stroke="black" points="190.784,-187.071 189.946,-197.632 197.188,-189.899 190.784,-187.071"/>
+</g>
+</g>
+</svg>

release-0.19.0/docs/devel/writing-a-getting-started-guide.md

+# Writing a Getting Started Guide
+This page gives some advice for anyone planning to write or update a Getting Started Guide for Kubernetes.
+It also gives some guidelines which reviewers should follow when reviewing a pull request for a
+guide. 
+
+A Getting Started Guide is instructions on how to create a Kubernetes cluster on top of a particular
+type(s) of infrastructure.  Infrastructure includes: the IaaS provider for VMs;
+the node OS; inter-node networking; and node Configuration Management system.
+A guide refers to scripts, Configuration Management files, and/or binary assets such as RPMs.  We call
+the combination of all these things needed to run on a particular type of infrastructure a
+**distro**.
+
+[The Matrix](../../docs/getting-started-guides/README.md) lists the distros.  If there is already a guide
+which is similar to the one you have planned, consider improving that one.
+
+
+Distros fall into two categories:  
+  - **versioned distros** are tested to work with a particular binary release of Kubernetes.  These
+    come in a wide variety, reflecting a wide range of ideas and preferences in how to run a cluster.
+  - **development distros** are tested work with the latest Kubernetes source code.  But, there are
+    relatively few of these and the bar is much higher for creating one.
+
+There are different guidelines for each.
+
+## Versioned Distro Guidelines
+These guidelines say *what* to do.  See the Rationale section for *why*.
+ - Send us a PR.
+ - Put the instructions in `docs/getting-started-guides/...`. Scripts go there too.  This helps devs easily
+   search for uses of flags by guides.
+ - We may ask that you host binary assets or large amounts of code in our `contrib` directory or on your
+   own repo.
+ - Setup a cluster and run the [conformance test](../../docs/devel/conformance-test.md) against it, and report the
+   results in your PR.
+ - Add or update a row in [The Matrix](../../docs/getting-started-guides/README.md).
+ - State the binary version of kubernetes that you tested clearly in your Guide doc and in The Matrix.
+ - Even if you are just updating the binary version used, please still do a conformance test.
+     - If it worked before and now fails, you can ask on IRC, 
+       check the release notes since your last tested version, or look at git -logs for files in other distros
+       that are updated to the new version.
+ - Versioned distros should typically not modify or add code in `cluster/`.  That is just scripts for developer
+   distros.  
+ - If a versioned distro has not been updated for many binary releases, it may be dropped from the Matrix.
+
+If you have a cluster partially working, but doing all the above steps seems like too much work,
+we still want to hear from you.  We suggest you write a blog post or a Gist, and we will link to it on our wiki page.
+Just file an issue or chat us on IRC and one of the committers will link to it from the wiki.
+
+## Development Distro Guidelines
+These guidelines say *what* to do.  See the Rationale section for *why*.
+  - the main reason to add a new development distro is to support a new IaaS provider (VM and
+    network management).  This means implementing a new `pkg/cloudprovider/$IAAS_NAME`.  
+  - Development distros should use Saltstack for Configuration Management.
+  - development distros need to support automated cluster creation, deletion, upgrading, etc.
+    This mean writing scripts in `cluster/$IAAS_NAME`.
+  - all commits to the tip of this repo need to not break any of the development distros
+    - the author of the change is responsible for making changes necessary on all the cloud-providers if the
+      change affects any of them, and reverting the change if it breaks any of the CIs.
+  - a development distro needs to have an organization which owns it.  This organization needs to: 
+    - Setting up and maintaining Continuous Integration that runs e2e frequently (multiple times per day) against the
+      Distro at head,  and which notifies all devs of breakage.
+    - being reasonably available for questions and assisting with
+      refactoring and feature additions that affect code for their IaaS.
+
+## Rationale 
+ - We want want people to create Kubernetes clusters with whatever IaaS, Node OS,
+   configuration management tools, and so on, which they are familiar with.  The
+   guidelines for **versioned distros** are designed for flexibility.
+ - We want developers to be able to work without understanding all the permutations of
+   IaaS, NodeOS, and configuration management.  The guidelines for **developer distros** are designed
+   for consistency.
+ - We want users to have a uniform experience with Kubernetes whenever they follow instructions anywhere
+   in our Github repository.  So, we ask that versioned distros pass a **conformance test** to make sure
+   really work.
+ - We ask versioned distros to **clearly state a version**.  People pulling from Github may 
+   expect any instructions there to work at Head, so stuff that has not been tested at Head needs
+   to be called out.  We are still changing things really fast, and, while the REST API is versioned,
+   it is not practical at this point to version or limit changes that affect distros.  We still change
+   flags at the Kubernetes/Infrastructure interface.
+ - We want to **limit the number of development distros** for several reasons.  Developers should
+   only have to change a limited number of places to add a new feature.  Also, since we will
+   gate commits on passing CI for all distros, and since end-to-end tests are typically somewhat
+   flaky, it would be highly likely for there to be false positives and CI backlogs with many CI pipelines.
+ - We do not require versioned distros to do **CI** for several reasons.  It is a steep
+   learning curve to understand our our automated testing scripts.  And it is considerable effort
+   to fully automate setup and teardown of a cluster, which is needed for CI.  And, not everyone
+   has the time and money to run CI.  We do not want to
+   discourage people from writing and sharing guides because of this.  
+ - Versioned distro authors are free to run their own CI and let us know if there is breakage, but we
+   will not include them as commit hooks -- there cannot be so many commit checks that it is impossible
+   to pass them all.
+ - We prefer a single Configuration Management tool for development distros.  If there were more
+   than one, the core developers would have to learn multiple tools and update config in multiple
+   places.  **Saltstack** happens to be the one we picked when we started the project.  We
+   welcome versioned distros that use any tool; there are already examples of 
+   CoreOS Fleet, Ansible, and others.
+ - You can still run code from head or your own branch
+   if you use another Configuration Management tool -- you just have to do some manual steps
+   during testing and deployment.
+      
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/writing-a-getting-started-guide.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/devel/writing-a-getting-started-guide.md?pixel)]()

release-0.19.0/docs/developer-guide.md

+# Kubernetes Developer Guide
+
+The developer guide is for anyone wanting to either write code which directly accesses the
+kubernetes API, or to contribute directly to the kubernetes project.
+It assumes some familiarity with concepts in the [User Guide](user-guide.md) and the [Cluster Admin
+Guide](cluster-admin-guide.md).
+
+
+## Developing against the Kubernetes API
+
+* API objects are explained at [http://kubernetes.io/third_party/swagger-ui/](http://kubernetes.io/third_party/swagger-ui/).
+
+* **Annotations** ([annotations.md](annotations.md)): are for attaching arbitrary non-identifying metadata to objects.
+  Programs that automate Kubernetes objects may use annotations to store small amounts of their state.
+
+* **API Conventions** ([api-conventions.md](api-conventions.md)):
+  Defining the verbs and resources used in the Kubernetes API.
+
+* **API Client Libraries** ([client-libraries.md](client-libraries.md)):
+  A list of existing client libraries, both supported and user-contributed.
+
+## Writing Plugins
+
+* **Authentication Plugins** ([authentication.md](authentication.md)):
+  The current and planned states of authentication tokens.
+
+* **Authorization Plugins** ([authorization.md](authorization.md)):
+  Authorization applies to all HTTP requests on the main apiserver port.
+  This doc explains the available authorization implementations.
+
+* **Admission Control Plugins** ([admission_control](design/admission_control.md))
+
+## Contributing to the Kubernetes Project
+
+See this [README](../docs/devel/README.md).
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/developer-guide.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/developer-guide.md?pixel)]()

release-0.19.0/docs/dns.md

+# DNS Integration with Kubernetes
+
+As of kubernetes 0.8, DNS is offered as a cluster add-on.  If enabled, a DNS
+Pod and Service will be scheduled on the cluster, and the kubelets will be
+configured to tell individual containers to use the DNS Service's IP.
+
+Every Service defined in the cluster (including the DNS server itself) will be
+assigned a DNS name.  By default, a client Pod's DNS search list will
+include the Pod's own namespace and the cluster's default domain.  This is best
+illustrated by example:
+
+Assume a Service named `foo` in the kubernetes namespace `bar`.  A Pod running
+in namespace `bar` can look up this service by simply doing a DNS query for
+`foo`.  A Pod running in namespace `quux` can look up this service by doing a
+DNS query for `foo.bar`.
+
+The cluster DNS server ([SkyDNS](https://github.com/skynetservices/skydns))
+supports forward lookups (A records) and service lookups (SRV records).
+
+## How it Works
+
+The DNS pod that runs holds 3 containers - skydns, etcd (which skydns uses),
+and a kubernetes-to-skydns bridge called kube2sky.  The kube2sky process
+watches the kubernetes master for changes in Services, and then writes the
+information to etcd, which skydns reads.  This etcd instance is not linked to
+any other etcd clusters that might exist, including the kubernetes master.
+
+## Issues
+
+The skydns service is reachable directly from kubernetes nodes (outside
+of any container) and DNS resolution works if the skydns service is targeted
+explicitly. However, nodes are not configured to use the cluster DNS service or
+to search the cluster's DNS domain by default.  This may be resolved at a later
+time.
+
+## For more information
+
+See [the docs for the cluster addon](../cluster/addons/dns/README.md).
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/dns.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/dns.md?pixel)]()

release-0.19.0/docs/downward_api.md

+# Downward API
+
+The downward API allows containers to consume information about the system without coupling to the
+kubernetes client or REST API.
+
+### Capabilities
+
+Containers can consume the following information via the downward API:
+
+*   Their pod's name
+*   Their pod's namespace
+
+### Consuming information about a pod in a container
+
+Containers consume information from the downward API using environment variables.  In the future,
+containers will also be able to consume the downward API via a volume plugin.  The `valueFrom`
+field of an environment variable allows you to specify an `ObjectFieldSelector` to select fields
+from the pod's definition.  The `ObjectFieldSelector` has an `apiVersion` field and a `fieldPath`
+field.  The `fieldPath` field is an expression designating a field on the pod.  The `apiVersion`
+field is the version of the API schema that the `fieldPath` is written in terms of.  If the
+`apiVersion` field is not specified it is defaulted to the API version of the enclosing object.
+
+### Example: consuming the downward API
+
+This is an example of a pod that consumes its name and namespace via the downward API:
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: dapi-test-pod
+spec:
+  containers:
+    - name: test-container
+      image: gcr.io/google_containers/busybox
+      command: [ "/bin/sh", "-c", "env" ]
+      env:
+        - name: POD_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: POD_NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+  restartPolicy: Never
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/downward_api.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/downward_api.md?pixel)]()

release-0.19.0/docs/getting-started-guides/README.md

+If you are not sure what OSes and infrastructure is supported, the table below lists all the combinations which have
+been tested recently.
+
+For the easiest "kick the tires" experience, please try the [local docker](docker.md) guide.
+
+If you are considering contributing a new guide, please read the
+[guidelines](../../docs/devel/writing-a-getting-started-guide.md).
+
+IaaS Provider  | Config. Mgmt | OS     | Networking  | Docs                                                   | Support Level                | Notes
+-------------- | ------------ | ------ | ----------  | ----------------------------------------------------   | ---------------------------- | -----
+GKE            |              |        | GCE         | [docs](https://cloud.google.com/container-engine)      | Commercial                   | Uses K8s version 0.15.0
+Vagrant        | Saltstack    | Fedora | OVS         | [docs](../../docs/getting-started-guides/vagrant.md)   | Project                      | Uses latest via https://get.k8s.io/
+GCE            | Saltstack    | Debian | GCE         | [docs](../../docs/getting-started-guides/gce.md)       | Project                      | Tested with 0.15.0 by @robertbailey
+Azure          | CoreOS       | CoreOS | Weave       | [docs](../../docs/getting-started-guides/coreos/azure/README.md)         | Community ([@errordeveloper](https://github.com/errordeveloper), [@squillace](https://github.com/squillace), [@chanezon](https://github.com/chanezon), [@crossorigin](https://github.com/crossorigin)) | Uses K8s version 0.17.0
+Docker Single Node | custom       | N/A    | local       | [docs](docker.md) | Project (@brendandburns) | Tested @ 0.14.1 |
+Docker Multi Node  | Flannel | N/A    | local       | [docs](docker-multinode.md) | Project (@brendandburns) | Tested @ 0.14.1 |
+Bare-metal     | Ansible      | Fedora | flannel     | [docs](../../docs/getting-started-guides/fedora/fedora_ansible_config.md)       | Project    | Uses K8s v0.13.2
+Bare-metal     | custom       | Fedora | _none_      | [docs](../../docs/getting-started-guides/fedora/fedora_manual_config.md) | Project    | Uses K8s v0.13.2
+Bare-metal     | custom       | Fedora | flannel   | [docs](../../docs/getting-started-guides/fedora/flannel_multi_node_cluster.md) | Community ([@aveshagarwal](https://github.com/aveshagarwal))| Tested with 0.15.0
+libvirt        | custom       | Fedora | flannel   | [docs](../../docs/getting-started-guides/fedora/flannel_multi_node_cluster.md) | Community ([@aveshagarwal](https://github.com/aveshagarwal))| Tested with 0.15.0
+KVM            | custom       | Fedora | flannel   | [docs](../../docs/getting-started-guides/fedora/flannel_multi_node_cluster.md) | Community ([@aveshagarwal](https://github.com/aveshagarwal))| Tested with 0.15.0
+Mesos/GCE      |              |        |             | [docs](../../docs/getting-started-guides/mesos.md)     | [Community](https://github.com/mesosphere/kubernetes-mesos) ([@jdef](https://github.com/jdef)) | Uses K8s v0.11.2
+AWS            | CoreOS       | CoreOS | flannel     | [docs](../../docs/getting-started-guides/coreos.md)    | Community                    | Uses K8s version 0.17.0
+GCE            | CoreOS       | CoreOS | flannel     | [docs](../../docs/getting-started-guides/coreos.md)    | Community (@kelseyhightower) | Uses K8s version 0.15.0
+Vagrant        | CoreOS       | CoreOS | flannel     | [docs](../../docs/getting-started-guides/coreos.md)    | Community ( [@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles) )           | Uses K8s version 0.15.0
+Bare-metal (Offline) | CoreOS       | CoreOS | flannel      | [docs](../../docs/getting-started-guides/coreos/bare_metal_offline.md) | Community([@jeffbean](https://github.com/jeffbean))    | Uses K8s version 0.15.0
+CloudStack     | Ansible      | CoreOS | flannel     | [docs](../../docs/getting-started-guides/cloudstack.md)| Community (@runseb)          | Uses K8s version 0.9.1
+Vmware         |              | Debian | OVS         | [docs](../../docs/getting-started-guides/vsphere.md)   | Community (@pietern)         | Uses K8s version 0.9.1
+Bare-metal     | custom       | CentOS | _none_      | [docs](../../docs/getting-started-guides/centos/centos_manual_config.md) | Community(@coolsvap)    | Uses K8s v0.9.1
+AWS            | Juju         | Ubuntu | flannel     | [docs](../../docs/getting-started-guides/juju.md)      | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) | [Tested](http://reports.vapour.ws/charm-tests-by-charm/kubernetes) K8s v0.8.1
+OpenStack/HPCloud | Juju      | Ubuntu | flannel     | [docs](../../docs/getting-started-guides/juju.md)      | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) | [Tested](http://reports.vapour.ws/charm-tests-by-charm/kubernetes) K8s v0.8.1
+Joyent         | Juju         | Ubuntu | flannel     | [docs](../../docs/getting-started-guides/juju.md)      | [Community](https://github.com/whitmo/bundle-kubernetes) ( [@whit](https://github.com/whitmo), [@matt](https://github.com/mbruzek), [@chuck](https://github.com/chuckbutler) ) | [Tested](http://reports.vapour.ws/charm-tests-by-charm/kubernetes) K8s v0.8.1
+AWS            | Saltstack    | Ubuntu | OVS         | [docs](../../docs/getting-started-guides/aws.md)       | Community (@justinsb)        | Uses K8s version 0.5.0
+Vmware         | CoreOS       | CoreOS | flannel     | [docs](../../docs/getting-started-guides/coreos.md)    | Community (@kelseyhightower) | Uses K8s version 0.15.0
+Azure          | Saltstack    | Ubuntu | OpenVPN     | [docs](../../docs/getting-started-guides/azure.md)     | Community                    |
+Bare-metal     | custom       | Ubuntu | flannel     | [docs](../../docs/getting-started-guides/ubuntu.md)    | Community (@resouer @WIZARD-CXY)       | use k8s version 0.18.0
+Docker Single Node        | custom       | N/A    | local       | [docs](docker.md) | Project (@brendandburns) | Tested @ 0.14.1 |
+Docker Multi Node        | Flannel| N/A    | local       | [docs](docker-multinode.md) | Project (@brendandburns) | Tested @ 0.14.1 |
+Local          |              |        | _none_      | [docs](../../docs/getting-started-guides/locally.md)   | Community (@preillyme)                     |
+libvirt/KVM    | CoreOS       | CoreOS | libvirt/KVM | [docs](../../docs/getting-started-guides/libvirt-coreos.md) | Community (@lhuard1A)   |
+oVirt          |              |        |             | [docs](../../docs/getting-started-guides/ovirt.md)     | Community (@simon3z)         |
+Rackspace      | CoreOS       | CoreOS | flannel     | [docs](../../docs/getting-started-guides/rackspace.md)     | Community (@doublerr)         | use k8s version 0.18.0
+
+
+*Note*: The above table is ordered by version test/used in notes followed by support level.
+
+Definition of columns:
+  - **IaaS Provider** is who/what provides the virtual or physical machines (nodes) that Kubernetes runs on.
+  - **OS** is the base operating system of the nodes.
+  - **Config. Mgmt** is the configuration management system that helps install and maintain kubernetes software on the
+    nodes.
+  - **Networking** is what implements the [networking model](../../docs/networking.md).  Those with networking type
+    _none_ may not support more than one node, or may support multiple VM nodes only in the same physical node.
+  - Support Levels
+    - **Project**:  Kubernetes Committers regularly use this configuration, so it usually works with the latest release
+      of Kubernetes.
+    - **Commercial**: A commercial offering with its own support arrangements.
+    - **Community**: Actively supported by community contributions. May not work with more recent releases of kubernetes.
+    - **Inactive**: No active maintainer.  Not recommended for first-time K8s users, and may be deleted soon.
+  - **Notes** is relevant information such as version k8s used.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/README.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/getting-started-guides/README.md?pixel)]()

release-0.19.0/docs/getting-started-guides/aws-coreos.md

+# Getting started on Amazon EC2 with CoreOS
+
+The example below creates an elastic Kubernetes cluster with a custom number of worker nodes and a master.
+
+**Warning:** contrary to the [supported procedure](aws.md), the examples below provision Kubernetes with an insecure API server (plain HTTP,
+no security tokens, no basic auth). For demonstration purposes only.
+
+## Highlights
+
+* Cluster bootstrapping using [cloud-config](https://coreos.com/docs/cluster-management/setup/cloudinit-cloud-config/)
+* Cross container networking with [flannel](https://github.com/coreos/flannel#flannel)
+* Auto worker registration with [kube-register](https://github.com/kelseyhightower/kube-register#kube-register)
+* Kubernetes v0.17.0 [official binaries](https://github.com/GoogleCloudPlatform/kubernetes/releases/tag/v0.17.0)
+
+## Prerequisites
+
+* [aws CLI](http://aws.amazon.com/cli)
+* [CoreOS image for AWS](https://coreos.com/docs/running-coreos/cloud-providers/ec2/)
+* [kubectl CLI](aws/kubectl.md)
+
+## Starting a Cluster
+
+### CloudFormation
+
+The [cloudformation-template.json](aws/cloudformation-template.json) can be used to bootstrap a Kubernetes cluster with a single command:
+
+```bash
+aws cloudformation create-stack --stack-name kubernetes --region us-west-2 \
+--template-body file://aws/cloudformation-template.json \
+--parameters ParameterKey=KeyPair,ParameterValue=<keypair> \
+             ParameterKey=ClusterSize,ParameterValue=<cluster_size> \
+             ParameterKey=VpcId,ParameterValue=<vpc_id> \
+             ParameterKey=SubnetId,ParameterValue=<subnet_id> \
+             ParameterKey=SubnetAZ,ParameterValue=<subnet_az>
+```
+
+It will take a few minutes for the entire stack to come up. You can monitor the stack progress with the following command:
+
+```bash
+aws cloudformation describe-stack-events --stack-name kubernetes
+```
+
+Record the Kubernetes Master IP address:
+
+```bash
+aws cloudformation describe-stacks --stack-name kubernetes
+```
+
+[Skip to kubectl client configuration](#configure-the-kubectl-ssh-tunnel)
+
+### AWS CLI
+
+The following commands shall use the latest CoreOS alpha AMI for the `us-west-2` region. For a list of different regions and corresponding AMI IDs see the [CoreOS EC2 cloud provider documentation](https://coreos.com/docs/running-coreos/cloud-providers/ec2/#choosing-a-channel).
+
+#### Create the Kubernetes Security Group
+
+```bash
+aws ec2 create-security-group --group-name kubernetes --description "Kubernetes Security Group"
+aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 22 --cidr 0.0.0.0/0
+aws ec2 authorize-security-group-ingress --group-name kubernetes --protocol tcp --port 80 --cidr 0.0.0.0/0
+aws ec2 authorize-security-group-ingress --group-name kubernetes --source-security-group-name kubernetes
+```
+
+#### Save the master and node cloud-configs
+
+* [master.yaml](aws/cloud-configs/master.yaml)
+* [node.yaml](aws/cloud-configs/node.yaml)
+
+#### Launch the master
+
+*Attention:* replace `<ami_image_id>` below for a [suitable version of CoreOS image for AWS](https://coreos.com/docs/running-coreos/cloud-providers/ec2/).
+
+```bash
+aws ec2 run-instances --image-id <ami_image_id> --key-name <keypair> \
+--region us-west-2 --security-groups kubernetes --instance-type m3.medium \
+--user-data file://master.yaml
+```
+
+Record the `InstanceId` for the master.
+
+Gather the public and private IPs for the master node:
+
+```bash
+aws ec2 describe-instances --instance-id <instance-id>
+```
+
+```
+{
+    "Reservations": [
+        {
+            "Instances": [
+                {
+                    "PublicDnsName": "ec2-54-68-97-117.us-west-2.compute.amazonaws.com", 
+                    "RootDeviceType": "ebs", 
+                    "State": {
+                        "Code": 16, 
+                        "Name": "running"
+                    }, 
+                    "PublicIpAddress": "54.68.97.117", 
+                    "PrivateIpAddress": "172.31.9.9", 
+...
+```
+
+#### Update the node.yaml cloud-config
+
+Edit `node.yaml` and replace all instances of `<master-private-ip>` with the **private** IP address of the master node.
+
+### Launch 3 worker nodes
+
+*Attention:* Replace `<ami_image_id>` below for a [suitable version of CoreOS image for AWS](https://coreos.com/docs/running-coreos/cloud-providers/ec2/#choosing-a-channel).
+
+```bash
+aws ec2 run-instances --count 3 --image-id <ami_image_id> --key-name <keypair> \
+--region us-west-2 --security-groups kubernetes --instance-type m3.medium \
+--user-data file://node.yaml
+```
+
+### Add additional worker nodes
+
+*Attention:* replace `<ami_image_id>` below for a [suitable version of CoreOS image for AWS](https://coreos.com/docs/running-coreos/cloud-providers/ec2/#choosing-a-channel).
+
+```bash
+aws ec2 run-instances --count 1 --image-id <ami_image_id> --key-name <keypair> \
+--region us-west-2 --security-groups kubernetes --instance-type m3.medium \
+--user-data file://node.yaml
+```
+
+### Configure the kubectl SSH tunnel
+
+This command enables secure communication between the kubectl client and the Kubernetes API.
+
+```bash
+ssh -f -nNT -L 8080:127.0.0.1:8080 core@<master-public-ip>
+```
+
+### Listing worker nodes
+
+Once the worker instances have fully booted, they will be automatically registered with the Kubernetes API server by the kube-register service running on the master node. It may take a few mins.
+
+```bash
+kubectl get nodes
+```
+
+## Starting a simple pod
+
+Create a pod manifest: `pod.json`
+
+```json
+{
+  "apiVersion": "v1",
+  "kind": "Pod",
+  "metadata": {
+    "name": "hello",
+    "labels": {
+      "name": "hello",
+      "environment": "testing"
+    }
+  },
+  "spec": {
+    "containers": [{
+      "name": "hello",
+      "image": "quay.io/kelseyhightower/hello",
+      "ports": [{
+        "containerPort": 80,
+        "hostPort": 80
+      }]
+    }]
+  }
+}
+```
+
+### Create the pod using the kubectl command line tool
+
+```bash
+kubectl create -f pod.json
+```
+
+### Testing
+
+```bash
+kubectl get pods
+```
+
+Record the **Host** of the pod, which should be the private IP address.
+
+Gather the public IP address for the worker node. 
+
+```bash
+aws ec2 describe-instances --filters 'Name=private-ip-address,Values=<host>'
+```
+
+```
+{
+    "Reservations": [
+        {
+            "Instances": [
+                {
+                    "PublicDnsName": "ec2-54-68-97-117.us-west-2.compute.amazonaws.com", 
+                    "RootDeviceType": "ebs", 
+                    "State": {
+                        "Code": 16, 
+                        "Name": "running"
+                    }, 
+                    "PublicIpAddress": "54.68.97.117", 
+...
+```
+
+Visit the public IP address in your browser to view the running pod.
+
+### Delete the pod
+
+```bash
+kubectl delete pods hello
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/aws-coreos.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/getting-started-guides/aws-coreos.md?pixel)]()

release-0.19.0/docs/getting-started-guides/aws.md

+# Getting started on AWS EC2
+
+## Prerequisites
+
+1. You need an AWS account. Visit [http://aws.amazon.com](http://aws.amazon.com) to get started
+2. Install and configure [AWS Command Line Interface](http://aws.amazon.com/cli)
+3. You need an AWS [instance profile and role](http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html) with EC2 full access.
+
+## Cluster turnup
+### Supported procedure: `get-kube`
+```bash
+#Using wget
+export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash
+
+#Using cURL
+export KUBERNETES_PROVIDER=aws; curl -sS https://get.k8s.io | bash
+```
+
+NOTE: This script calls [cluster/kube-up.sh](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/cluster/kube-up.sh)
+which in turn calls [cluster/aws/util.sh](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/cluster/aws/util.sh)
+using [cluster/aws/config-default.sh](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/cluster/aws/config-default.sh).
+
+This process takes about 5 to 10 minutes. Once the cluster is up, the IP addresses of your master and node(s) will be printed,
+as well as information about the default services running in the cluster (monitoring, logging, dns). User credentials and security
+tokens are written in `~/.kube/kubeconfig`, they will be necessary to use the CLI or the HTTP Basic Auth.
+
+By default, the script will provision a new VPC and a 4 node k8s cluster in us-west-2a (Oregon) with `t2.micro` instances running on Ubuntu.
+You can override the variables defined in [config-default.sh](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/cluster/aws/config-default.sh) to change this behavior as follows:
+
+```bash
+export KUBE_AWS_ZONE=eu-west-1c
+export NUM_MINIONS=2
+export MINION_SIZE=m3.medium
+export AWS_S3_REGION=eu-west-1
+export AWS_S3_BUCKET=mycompany-kubernetes-artifacts
+export INSTANCE_PREFIX=k8s
+...
+```
+
+It will also try to create or reuse a keypair called "kubernetes", and IAM profiles called "kubernetes-master" and "kubernetes-minion".
+If these already exist, make sure you want them to be used here.
+
+NOTE: If using an existing keypair named "kubernetes" then you must set the `AWS_SSH_KEY` key to point to your private key.
+
+### Alternatives
+A contributed [example](aws-coreos.md) allows you to setup a Kubernetes cluster based on [CoreOS](http://www.coreos.com), either using
+AWS CloudFormation or EC2 with user data (cloud-config).
+
+## Getting started with your cluster
+### Command line administration tool: `kubectl`
+Copy the appropriate `kubectl` binary to any location defined in your `PATH` environment variable, for example:
+
+```bash
+# OS X
+sudo cp kubernetes/platforms/darwin/amd64/kubectl /usr/local/bin/kubectl
+
+# Linux
+sudo cp kubernetes/platforms/linux/amd64/kubectl /usr/local/bin/kubectl
+```
+
+An up-to-date documentation page for this tool is available here: [kubectl manual](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/kubectl.md)
+
+By default, `kubectl` will use the `kubeconfig` file generated during the cluster startup for authenticating against the API.
+For more information, please read [kubeconfig files](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/kubeconfig-file.md)
+
+### Examples
+See [a simple nginx example](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/simple-nginx.md) to try out your new cluster.
+
+The "Guestbook" application is another popular example to get started with Kubernetes: [guestbook example](https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/guestbook)
+
+For more complete applications, please look in the [examples directory](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples)
+
+## Tearing down the cluster
+Make sure the environment variables you used to provision your cluster are still exported, then call the following script inside the
+`kubernetes` directory:
+
+```bash
+cluster/kube-down.sh
+```
+
+## Further reading
+Please see the [Kubernetes docs](https://github.com/GoogleCloudPlatform/kubernetes/tree/master/docs) for more details on administering
+and using a Kubernetes cluster.
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/aws.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/getting-started-guides/aws.md?pixel)]()

release-0.19.0/docs/getting-started-guides/aws/cloud-configs/master.yaml

+#cloud-config
+
+write_files:
+  - path: /opt/bin/waiter.sh
+    owner: root
+    permissions: 0755
+    content: |
+      #! /usr/bin/bash
+      until curl http://127.0.0.1:2379/v2/machines; do sleep 2; done
+
+coreos:
+  etcd2:
+    name: master
+    initial-cluster-token: k8s_etcd
+    initial-cluster: master=http://$private_ipv4:2380
+    listen-peer-urls: http://$private_ipv4:2380,http://localhost:2380
+    initial-advertise-peer-urls: http://$private_ipv4:2380
+    listen-client-urls: http://$private_ipv4:2379,http://localhost:2379
+    advertise-client-urls: http://$private_ipv4:2379
+  fleet:
+    etcd_servers: http://localhost:2379
+    metadata: k8srole=master
+  flannel:
+    etcd_endpoints: http://localhost:2379
+  locksmithd:
+    endpoint: http://localhost:2379
+  units:
+    - name: etcd2.service
+      command: start
+    - name: fleet.service
+      command: start
+    - name: etcd2-waiter.service
+      command: start
+      content: |
+        [Unit]
+        Description=etcd waiter
+        Wants=network-online.target
+        Wants=etcd2.service
+        After=etcd2.service
+        After=network-online.target
+        Before=flanneld.service fleet.service locksmithd.service
+
+        [Service]
+        ExecStart=/usr/bin/bash /opt/bin/waiter.sh
+        RemainAfterExit=true
+        Type=oneshot
+    - name: flanneld.service
+      command: start
+      drop-ins:
+        - name: 50-network-config.conf
+          content: |
+            [Service]
+            ExecStartPre=-/usr/bin/etcdctl mk /coreos.com/network/config '{"Network": "10.244.0.0/16", "Backend": {"Type": "vxlan"}}'
+    - name: docker-cache.service
+      command: start
+      content: |
+        [Unit]
+        Description=Docker cache proxy
+        Requires=early-docker.service
+        After=early-docker.service
+        Before=early-docker.target
+
+        [Service]
+        Restart=always
+        TimeoutStartSec=0
+        RestartSec=5
+        Environment=TMPDIR=/var/tmp/
+        Environment=DOCKER_HOST=unix:///var/run/early-docker.sock
+        ExecStartPre=-/usr/bin/docker kill docker-registry
+        ExecStartPre=-/usr/bin/docker rm docker-registry
+        ExecStartPre=/usr/bin/docker pull quay.io/devops/docker-registry:latest
+        # GUNICORN_OPTS is an workaround for
+        # https://github.com/docker/docker-registry/issues/892
+        ExecStart=/usr/bin/docker run --rm --net host --name docker-registry \
+            -e STANDALONE=false \
+            -e GUNICORN_OPTS=[--preload] \
+            -e MIRROR_SOURCE=https://registry-1.docker.io \
+            -e MIRROR_SOURCE_INDEX=https://index.docker.io \
+            -e MIRROR_TAGS_CACHE_TTL=1800 \
+            quay.io/devops/docker-registry:latest
+    - name: docker.service
+      drop-ins:
+        - name: 51-docker-mirror.conf
+          content: |
+            [Unit]
+            # making sure that docker-cache is up and that flanneld finished
+            # startup, otherwise containers won't land in flannel's network...
+            Requires=docker-cache.service
+            After=docker-cache.service
+
+            [Service]
+            Environment=DOCKER_OPTS='--registry-mirror=http://$private_ipv4:5000'
+    - name: get-kubectl.service
+      command: start
+      content: |
+        [Unit]
+        Description=Get kubectl client tool
+        Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+        Requires=network-online.target
+        After=network-online.target
+
+        [Service]
+        ExecStart=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kubectl
+        ExecStart=/usr/bin/chmod +x /opt/bin/kubectl
+        Type=oneshot
+        RemainAfterExit=true
+    - name: kube-apiserver.service
+      command: start
+      content: |
+        [Unit]
+        Description=Kubernetes API Server
+        Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+        Requires=etcd2-waiter.service
+        After=etcd2-waiter.service
+
+        [Service]
+        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kube-apiserver
+        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-apiserver
+        ExecStart=/opt/bin/kube-apiserver \
+        --insecure-bind-address=0.0.0.0 \
+        --service-cluster-ip-range=10.100.0.0/16 \
+        --etcd-servers=http://localhost:2379
+        Restart=always
+        RestartSec=10
+    - name: kube-controller-manager.service
+      command: start
+      content: |
+        [Unit]
+        Description=Kubernetes Controller Manager
+        Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+        Requires=kube-apiserver.service
+        After=kube-apiserver.service
+
+        [Service]
+        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kube-controller-manager
+        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-controller-manager
+        ExecStart=/opt/bin/kube-controller-manager \
+        --master=127.0.0.1:8080
+        Restart=always
+        RestartSec=10
+    - name: kube-scheduler.service
+      command: start
+      content: |
+        [Unit]
+        Description=Kubernetes Scheduler
+        Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+        Requires=kube-apiserver.service
+        After=kube-apiserver.service
+
+        [Service]
+        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kube-scheduler
+        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-scheduler
+        ExecStart=/opt/bin/kube-scheduler \
+        --master=127.0.0.1:8080
+        Restart=always
+        RestartSec=10
+    - name: kube-register.service
+      command: start
+      content: |
+        [Unit]
+        Description=Kubernetes Registration Service
+        Documentation=https://github.com/kelseyhightower/kube-register
+        Requires=kube-apiserver.service fleet.service
+        After=kube-apiserver.service fleet.service
+
+        [Service]
+        ExecStartPre=-/usr/bin/wget -nc -O /opt/bin/kube-register https://github.com/kelseyhightower/kube-register/releases/download/v0.0.3/kube-register-0.0.3-linux-amd64
+        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-register
+        ExecStart=/opt/bin/kube-register \
+        --metadata=k8srole=node \
+        --fleet-endpoint=unix:///var/run/fleet.sock \
+        --api-endpoint=http://127.0.0.1:8080
+        Restart=always
+        RestartSec=10
+  update:
+    group: alpha
+    reboot-strategy: off

release-0.19.0/docs/getting-started-guides/aws/cloud-configs/node.yaml

+#cloud-config
+
+write_files:
+  - path: /opt/bin/wupiao
+    owner: root
+    permissions: 0755
+    content: |
+      #!/bin/bash
+      # [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen
+      [ -n "$1" ] && [ -n "$2" ] && while ! curl --output /dev/null \
+        --silent --head --fail \
+        http://${1}:${2}; do sleep 1 && echo -n .; done;
+      exit $?
+
+coreos:
+  etcd2:
+    listen-client-urls: http://localhost:2379
+    advertise-client-urls: http://0.0.0.0:2379
+    initial-cluster: master=http://<master-private-ip>:2380
+    proxy: on
+  fleet:
+    etcd_servers: http://localhost:2379
+    metadata: k8srole=node
+  flannel:
+    etcd_endpoints: http://localhost:2379
+  locksmithd:
+    endpoint: http://localhost:2379
+  units:
+    - name: etcd2.service
+      command: start
+    - name: fleet.service
+      command: start
+    - name: flanneld.service
+      command: start
+    - name: docker.service
+      command: start
+      drop-ins:
+        - name: 50-docker-mirror.conf
+          content: |
+            [Service]
+            Environment=DOCKER_OPTS='--registry-mirror=http://<master-private-ip>:5000'
+    - name: kubelet.service
+      command: start
+      content: |
+        [Unit]
+        Description=Kubernetes Kubelet
+        Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+        Requires=network-online.target
+        After=network-online.target
+
+        [Service]
+        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kubelet
+        ExecStartPre=/usr/bin/chmod +x /opt/bin/kubelet
+        # wait for kubernetes master to be up and ready
+        ExecStartPre=/opt/bin/wupiao <master-private-ip> 8080
+        ExecStart=/opt/bin/kubelet \
+        --api-servers=<master-private-ip>:8080 \
+        --hostname-override=$private_ipv4
+        Restart=always
+        RestartSec=10
+    - name: kube-proxy.service
+      command: start
+      content: |
+        [Unit]
+        Description=Kubernetes Proxy
+        Documentation=https://github.com/GoogleCloudPlatform/kubernetes
+        Requires=network-online.target
+        After=network-online.target
+
+        [Service]
+        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kube-proxy
+        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-proxy
+        # wait for kubernetes master to be up and ready
+        ExecStartPre=/opt/bin/wupiao <master-private-ip> 8080
+        ExecStart=/opt/bin/kube-proxy \
+        --master=http://<master-private-ip>:8080
+        Restart=always
+        RestartSec=10
+  update:
+    group: alpha
+    reboot-strategy: off

release-0.19.0/docs/getting-started-guides/aws/kubectl.md

+# Install and configure kubectl
+
+## Download the kubectl CLI tool
+```bash
+### Darwin
+wget https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/darwin/amd64/kubectl
+
+### Linux
+wget https://storage.googleapis.com/kubernetes-release/release/v0.17.0/bin/linux/amd64/kubectl
+```
+
+### Copy kubectl to your path
+```bash
+chmod +x kubectl
+mv kubectl /usr/local/bin/
+```
+
+### Create a secure tunnel for API communication
+```bash
+ssh -f -nNT -L 8080:127.0.0.1:8080 core@<master-public-ip>
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/aws/kubectl.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/getting-started-guides/aws/kubectl.md?pixel)]()

release-0.19.0/docs/getting-started-guides/azure.md

+## Getting started on Microsoft Azure
+
+### Azure Prerequisites
+
+1. You need an Azure account. Visit http://azure.microsoft.com/ to get started.
+2. Install and configure the Azure cross-platform command-line interface. http://azure.microsoft.com/en-us/documentation/articles/xplat-cli/
+3. Make sure you have a default account set in the Azure cli, using `azure account set`
+
+### Prerequisites for your workstation
+
+1. Be running a Linux or Mac OS X.
+2. Get or build a [binary release](binary_release.md)
+3. If you want to build your own release, you need to have [Docker
+installed](https://docs.docker.com/installation/).  On Mac OS X you can use
+[boot2docker](http://boot2docker.io/).
+
+### Setup
+The cluster setup scripts can setup Kubernetes for multiple targets. First modify `cluster/kube-env.sh` to specify azure:
+
+    KUBERNETES_PROVIDER="azure"
+
+Next, specify an existing virtual network and subnet in `cluster/azure/config-default.sh`:
+
+    AZ_VNET=<vnet name>
+    AZ_SUBNET=<subnet name>
+
+You can create a virtual network:
+
+    azure network vnet create <vnet name> --subnet=<subnet name> --location "West US" -v
+
+Now you're ready.
+
+You can then use the `cluster/kube-*.sh` scripts to manage your azure cluster, start with:
+
+    cluster/kube-up.sh
+
+The script above will start (by default) a single master VM along with 4 worker VMs.  You
+can tweak some of these parameters by editing `cluster/azure/config-default.sh`.
+
+### Getting started with your cluster
+See [a simple nginx example](../../examples/simple-nginx.md) to try out your new cluster.
+
+For more complete applications, please look in the [examples directory](../../examples).
+
+### Tearing down the cluster
+```
+cluster/kube-down.sh
+```
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/azure.md?pixel)]()
+
+
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/release-0.19.0/docs/getting-started-guides/azure.md?pixel)]()