This document describes the environment for Kubelet managed containers on a Kubernetes node (kNode). In contrast to the Kubernetes cluster API, which provides an API for creating and managing containers, the Kubernetes container environment provides the container access to information about what else is going on in the cluster.
This document describes the environment for Kubelet managed containers on a Kubernetes node (kNode). In contrast to the Kubernetes cluster API, which provides an API for creating and managing containers, the Kubernetes container environment provides the container access to information about what else is going on in the cluster.
This cluster information makes it possible to build applications that are *cluster aware*.
This cluster information makes it possible to build applications that are *cluster aware*.
Additionally, the Kubernetes container environment defines a series of signals that are surfaced to optional signal handlers defined as part of individual containers. Container signals are somewhat analagous to operating system signals in a traditional process model. However these signals are designed to make it easier to build reliable, scalable cloud applications in the Kubernetes cluster. Containers that participate in this cluster lifecycle become *cluster native*.
Additionally, the Kubernetes container environment defines a series of hooks that are surfaced to optional hook handlers defined as part of individual containers. Container hooks are somewhat analagous to operating system signals in a traditional process model. However these hooks are designed to make it easier to build reliable, scalable cloud applications in the Kubernetes cluster. Containers that participate in this cluster lifecycle become *cluster native*.
Another important part of the container environment is the file system that is available to the container. In Kubernetes, the filesystem is a combination of an [image](./images.md) and one or more [volumes](./volumes.md).
Another important part of the container environment is the file system that is available to the container. In Kubernetes, the filesystem is a combination of an [image](./images.md) and one or more [volumes](./volumes.md).
The following sections describe both the cluster information provided to containers, as well as the signals and life-cycle that allows containers to interact with the management system.
The following sections describe both the cluster information provided to containers, as well as the hooks and life-cycle that allows containers to interact with the management system.
## Cluster Information
## Cluster Information
There are two types of information that are available within the container environment. There is information about the container itself, and there is information about other objects in the system.
There are two types of information that are available within the container environment. There is information about the container itself, and there is information about other objects in the system.
...
@@ -32,29 +32,29 @@ FOO_SERVICE_PORT=<the port the service is running on>
...
@@ -32,29 +32,29 @@ FOO_SERVICE_PORT=<the port the service is running on>
Going forward, we expect that Services will have a dedicated IP address. In that context, we will also surface services to the container via DNS. Of course DNS is still not an enumerable protocol, so we will continue to provide environment variables so that containers can do discovery.
Going forward, we expect that Services will have a dedicated IP address. In that context, we will also surface services to the container via DNS. Of course DNS is still not an enumerable protocol, so we will continue to provide environment variables so that containers can do discovery.
## Container Signals
## Container Hooks
*NB*: Container signals are under active development, we anticipate adding additional signals as the Kubernetes container management system evolves.*
*NB*: Container hooks are under active development, we anticipate adding additional hooks as the Kubernetes container management system evolves.*
Container signals provide information to the container about events in its management lifecycle. For example, immediately after a container is started, it receives a *PostStart* signal. These signals are broadcast *into* the container with information about the life-cycle of the container. They are different from the events provided by Docker and other systems which are *output* from the container. Output events provide a log of what has already happened. Input signals provide real-time notification about things that are happening, but no historical log.
Container hooks provide information to the container about events in its management lifecycle. For example, immediately after a container is started, it receives a *PostStart* hook. These hooks are broadcast *into* the container with information about the life-cycle of the container. They are different from the events provided by Docker and other systems which are *output* from the container. Output events provide a log of what has already happened. Input hooks provide real-time notification about things that are happening, but no historical log.
### Signal Details
### Hook Details
There are currently two container signals that are surfaced to containers, and two proposed signals:
There are currently two container hooks that are surfaced to containers, and two proposed hooks:
*PreStart - ****Proposed***
*PreStart - ****Proposed***
This signal is sent immediately before a container is created. It signals that the container will be created immediately after the call completes. No parameters are passed. *Note - *Some event handlers (namely ‘exec’ are incompatible with this event)
This hook is sent immediately before a container is created. It notifies that the container will be created immediately after the call completes. No parameters are passed. *Note - *Some event handlers (namely ‘exec’ are incompatible with this event)
*PostStart*
*PostStart*
This signal is sent immediately after a container is created. It signals to the container that it has been created. No parameters are passed to the handler.
This hook is sent immediately after a container is created. It notifies the container that it has been created. No parameters are passed to the handler.
*PostRestart - ****Proposed***
*PostRestart - ****Proposed***
This signal is called before the PostStart handler, when a container has been restarted, rather than started for the first time. No parameters are passed to the handler.
This hook is called before the PostStart handler, when a container has been restarted, rather than started for the first time. No parameters are passed to the handler.
*PreStop*
*PreStop*
This signal is called immediately before a container is terminated. This event handler is blocking, and must complete before the call to delete the container is sent to the Docker daemon. The SIGTERM notification sent by Docker is also still sent.
This hook is called immediately before a container is terminated. This event handler is blocking, and must complete before the call to delete the container is sent to the Docker daemon. The SIGTERM notification sent by Docker is also still sent.
A single parameter named reason is passed to the handler which contains the reason for termination. Currently the valid values for reason are:
A single parameter named reason is passed to the handler which contains the reason for termination. Currently the valid values for reason are:
* ● ```Delete``` - indicating an API call to delete the pod containing this container.
* ● ```Delete``` - indicating an API call to delete the pod containing this container.
...
@@ -64,13 +64,13 @@ A single parameter named reason is passed to the handler which contains the reas
...
@@ -64,13 +64,13 @@ A single parameter named reason is passed to the handler which contains the reas
Eventually, user specified reasons may be [added to the API](https://github.com/GoogleCloudPlatform/kubernetes/issues/137).
Eventually, user specified reasons may be [added to the API](https://github.com/GoogleCloudPlatform/kubernetes/issues/137).
### Signal Handler Execution
### Hook Handler Execution
When a management signal occurs, the management system calls into any registered signal handlers in the container for that signal. These signal handler calls are synchronous in the context of the pod containing the container. Note:this means that signal handler execution blocks any further management of the pod. If your signal handler blocks, no other management (including health checks) will occur until the signal handler completes. Blocking signal handlers do *not* affect management of other Pods. Typically we expect that users will make their signal handlers as lightweight as possible, but there are cases where long running commands make sense (e.g. saving state prior to container stop)
When a management hook occurs, the management system calls into any registered hook handlers in the container for that hook. These hook handler calls are synchronous in the context of the pod containing the container. Note:this means that hook handler execution blocks any further management of the pod. If your hook handler blocks, no other management (including health checks) will occur until the hook handler completes. Blocking hook handlers do *not* affect management of other Pods. Typically we expect that users will make their hook handlers as lightweight as possible, but there are cases where long running commands make sense (e.g. saving state prior to container stop)
For signals which have parameters, these parameters are passed to the event handler as a set of key/value pairs. The details of this parameter passing is handler implementation dependent (see below)
For hooks which have parameters, these parameters are passed to the event handler as a set of key/value pairs. The details of this parameter passing is handler implementation dependent (see below)
### Signal Handler Implementations
### Hook Handler Implementations
Signal handlers are the way that signals are surfaced to containers. Containers can select the type of signal handler they would like to implement. Kubernetes currently supports two different signal handler types:
Hook handlers are the way that hooks are surfaced to containers. Containers can select the type of hook handler they would like to implement. Kubernetes currently supports two different hook handler types:
* Exec - Executes a specific command (e.g. pre-stop.sh) inside the cgroup and namespaces of the container. Resources consumed by the command are counted against the container. Commands which return non-zero values are treated as container failures (and will cause kubelet to forcibly restart the container). Parameters are passed to the command as traditional linux command line flags (e.g. pre-stop.sh --reason=HEALTH)
* Exec - Executes a specific command (e.g. pre-stop.sh) inside the cgroup and namespaces of the container. Resources consumed by the command are counted against the container. Commands which return non-zero values are treated as container failures (and will cause kubelet to forcibly restart the container). Parameters are passed to the command as traditional linux command line flags (e.g. pre-stop.sh --reason=HEALTH)
In Kubernetes, rather than individual containers, _pods_ are the smallest deployable units that can be created, scheduled, and managed. Singleton pods can be created directly, and sets of pods may created, maintained, and scaled using replication controllers. Services create load-balanced targets for sets of pods.
The Kubernetes API currently manages 3 main resources: `pods`,
`replicationControllers`, and `services`. Pods correspond to colocated groups
of [Docker containers](http://docker.io) with shared volumes, as supported by
can be created directly via the `/pods` endpoint. Sets of pods may created,
maintained, and scaled using replicationControllers. Services create
load-balanced targets for sets of pods.
Each resource has a two [identifiers](identifiers.md): a string `Name` and a
string `UID`. The name is provided by the user. The UID is generated by the
system and is guaranteed to be unique in space and time across all resources.
`labels` is a map of string (key) to string (value).
Each resource has list of key-value [labels](labels.md).
Individual labels are used to specify identifying metadata that can be used to define sets of resources by
specifying required labels.
## Creation and Updates
Object creation is idempotent when the client remembers the name of the object it wants to create.
Resources have a `desiredState` for the user provided parameters and a
`currentState` for the actual system state. When a new version of a resource
is PUT the `desiredState` is updated and available immediately. Over time the
system will work to bring the `currentState` into line with the `desiredState`.
The system will drive toward the most recent `desiredState` regardless of
previous versions of that stanza. In other words, if a value is changed from 2
to 5 in one PUT and then back down to 3 in another PUT the system is not
required to 'touch base' at 5 before making 3 the `currentState`.
When doing an update, we assume that the entire `desiredState` stanza is
specified. If a field is omitted it is assumed that the user is looking to
delete that field. It is viable for a user to GET the resource, modify what
they like in the `desiredState` or labels stanzas and then PUT it back. If the
`currentState` is included in the PUT it will be silently ignored.
Concurrent modification should be accomplished with optimistic locking of
resources. All resources have a `ResourceVersion` as part of their metadata.
If this is included with the PUT operation the system will verify that there
have not been other successful mutations to the resource during a
read/modify/write cycle. The correct client action at this point is to GET the
resource again, apply the changes afresh and try submitting again.
Kubernetes supports a unique [networking model](networking.md). Kubernetes encourages a flat address and does not dynamically allocate ports, instead allowing users to select whichever ports are convenient for them. To achieve this, it allocates an IP address for each pod and each service. Services provide stable addresses and [DNS names](dns.md) for clients to connect to, even as serving pods are replaced by new pods on new hosts.
Each resource has a map of key-value [labels](labels.md). Individual labels are used to specify identifying metadata that can be used to define sets of resources by specifying required labels.
Each resource also has a map of string keys and values that can be used by external tooling to store and retrieve arbitrary metadata about this object, called [annotations.md](annotations.md).
Each resource is created within a specific [namespace](namespaces.md), a default one if unspecified.
In Kubernetes, rather than individual containers, _pods_ are the smallest deployable units that can be created, scheduled, and managed.
## What is a _pod_?
## What is a _pod_?
A _pod_ (as in a pod of whales or pea pod) models an application-specific "logical host" in a containerized environment. It may contain one or more containers which are relatively tightly coupled -- in a pre-container world, they would have executed on the same physical or virtual host.
A _pod_ (as in a pod of whales or pea pod) correspond to a colocated group of [Docker containers](http://docker.io) with shared [volumes](volumes.md). A pod models an application-specific "logical host" in a containerized environment. It may contain one or more containers which are relatively tightly coupled -- in a pre-container world, they would have executed on the same physical or virtual host.
Like running containers, pods are considered to be relatively ephemeral rather than durable entities. As discussed in [life of a pod](pod-states.md), pods are scheduled to nodes and remain there until termination (according to restart policy) or deletion. When a node dies, the pods scheduled to that node are deleted. Specific pods are never rescheduled to new nodes; instead, they must be replaced (see [replication controller](replication-controller.md) for more details). (In the future, a higher-level API may support pod migration.)
Like running containers, pods are considered to be relatively ephemeral rather than durable entities. As discussed in [life of a pod](pod-states.md), pods are scheduled to nodes and remain there until termination (according to restart policy) or deletion. When a node dies, the pods scheduled to that node are deleted. Specific pods are never rescheduled to new nodes; instead, they must be replaced (see [replication controller](replication-controller.md) for more details). (In the future, a higher-level API may support pod migration.)
