@@ -118,7 +118,7 @@ In order to preserve extensibility, in the future, we intend to explicitly conve
...
@@ -118,7 +118,7 @@ In order to preserve extensibility, in the future, we intend to explicitly conve
Note that historical information status (e.g., last transition time, failure counts) is only provided at best effort, and is not guaranteed to not be lost.
Note that historical information status (e.g., last transition time, failure counts) is only provided at best effort, and is not guaranteed to not be lost.
Status information that may be large (especially unbounded in size, such as lists of references to other objects -- see below) and/or rapidly changing, such as [resource usage](./design/resources.md#usage-data), should be put into separate objects, with possibly a reference from the original object. This helps to ensure that GETs and watch remain reasonably efficient for the majority of clients, which may not need that data.
Status information that may be large (especially unbounded in size, such as lists of references to other objects -- see below) and/or rapidly changing, such as [resource usage](design/resources.md#usage-data), should be put into separate objects, with possibly a reference from the original object. This helps to ensure that GETs and watch remain reasonably efficient for the majority of clients, which may not need that data.
@@ -7,7 +7,7 @@ This document describes the environment for Kubelet managed containers on a Kube
...
@@ -7,7 +7,7 @@ This document describes the environment for Kubelet managed containers on a Kube
This cluster information makes it possible to build applications that are *cluster aware*.
This cluster information makes it possible to build applications that are *cluster aware*.
Additionally, the Kubernetes container environment defines a series of hooks that are surfaced to optional hook handlers defined as part of individual containers. Container hooks are somewhat analogous to operating system signals in a traditional process model. However these hooks are designed to make it easier to build reliable, scalable cloud applications in the Kubernetes cluster. Containers that participate in this cluster lifecycle become *cluster native*.
Additionally, the Kubernetes container environment defines a series of hooks that are surfaced to optional hook handlers defined as part of individual containers. Container hooks are somewhat analogous to operating system signals in a traditional process model. However these hooks are designed to make it easier to build reliable, scalable cloud applications in the Kubernetes cluster. Containers that participate in this cluster lifecycle become *cluster native*.
Another important part of the container environment is the file system that is available to the container. In Kubernetes, the filesystem is a combination of an [image](./images.md) and one or more [volumes](./volumes.md).
Another important part of the container environment is the file system that is available to the container. In Kubernetes, the filesystem is a combination of an [image](images.md) and one or more [volumes](volumes.md).
The following sections describe both the cluster information provided to containers, as well as the hooks and life-cycle that allows containers to interact with the management system.
The following sections describe both the cluster information provided to containers, as well as the hooks and life-cycle that allows containers to interact with the management system.
K8s will have a have a `namespace` API object. It is similar to a Google Compute Engine `project`. It provides a namespace for objects created by a group of people co-operating together, preventing name collisions with non-cooperating groups. It also serves as a reference point for authorization policies.
K8s will have a have a `namespace` API object. It is similar to a Google Compute Engine `project`. It provides a namespace for objects created by a group of people co-operating together, preventing name collisions with non-cooperating groups. It also serves as a reference point for authorization policies.
Namespaces are described in [namespace.md](namespaces.md).
Namespaces are described in [namespaces.md](namespaces.md).
In the Enterprise Profile:
In the Enterprise Profile:
- a `userAccount` may have permission to access several `namespace`s.
- a `userAccount` may have permission to access several `namespace`s.
...
@@ -151,7 +151,7 @@ In the Simple Profile:
...
@@ -151,7 +151,7 @@ In the Simple Profile:
Namespaces versus userAccount vs Labels:
Namespaces versus userAccount vs Labels:
-`userAccount`s are intended for audit logging (both name and UID should be logged), and to define who has access to `namespace`s.
-`userAccount`s are intended for audit logging (both name and UID should be logged), and to define who has access to `namespace`s.
-`labels` (see [docs/labels.md](/docs/labels.md)) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities.
-`labels` (see [docs/labels.md](../../docs/labels.md)) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities.
-`namespace`s prevent name collisions between uncoordinated groups of people, and provide a place to attach common policies for co-operating groups of people.
-`namespace`s prevent name collisions between uncoordinated groups of people, and provide a place to attach common policies for co-operating groups of people.
@@ -149,7 +149,7 @@ The following are planned future extensions to the resource model, included here
...
@@ -149,7 +149,7 @@ The following are planned future extensions to the resource model, included here
## Usage data
## Usage data
Because resource usage and related metrics change continuously, need to be tracked over time (i.e., historically), can be characterized in a variety of ways, and are fairly voluminous, we will not include usage in core API objects, such as [Pods](pods.md) and Nodes, but will provide separate APIs for accessing and managing that data. See the Appendix for possible representations of usage data, but the representation we'll use is TBD.
Because resource usage and related metrics change continuously, need to be tracked over time (i.e., historically), can be characterized in a variety of ways, and are fairly voluminous, we will not include usage in core API objects, such as [Pods](../pods.md) and Nodes, but will provide separate APIs for accessing and managing that data. See the Appendix for possible representations of usage data, but the representation we'll use is TBD.
Singleton values for observed and predicted future usage will rapidly prove inadequate, so we will support the following structure for extended usage information:
Singleton values for observed and predicted future usage will rapidly prove inadequate, so we will support the following structure for extended usage information:
@@ -63,14 +63,14 @@ Automated process users fall into the following categories:
...
@@ -63,14 +63,14 @@ Automated process users fall into the following categories:
A pod runs in a *security context* under a *service account* that is defined by an administrator or project administrator, and the *secrets* a pod has access to is limited by that *service account*.
A pod runs in a *security context* under a *service account* that is defined by an administrator or project administrator, and the *secrets* a pod has access to is limited by that *service account*.
1. The API should authenticate and authorize user actions [authn and authz](./access.md)
1. The API should authenticate and authorize user actions [authn and authz](access.md)
2. All infrastructure components (kubelets, kube-proxies, controllers, scheduler) should have an infrastructure user that they can authenticate with and be authorized to perform only the functions they require against the API.
2. All infrastructure components (kubelets, kube-proxies, controllers, scheduler) should have an infrastructure user that they can authenticate with and be authorized to perform only the functions they require against the API.
3. Most infrastructure components should use the API as a way of exchanging data and changing the system, and only the API should have access to the underlying data store (etcd)
3. Most infrastructure components should use the API as a way of exchanging data and changing the system, and only the API should have access to the underlying data store (etcd)
4. When containers run on the cluster and need to talk to other containers or the API server, they should be identified and authorized clearly as an autonomous process via a [service account](./service_accounts.md)
4. When containers run on the cluster and need to talk to other containers or the API server, they should be identified and authorized clearly as an autonomous process via a [service account](service_accounts.md)
1. If the user who started a long-lived process is removed from access to the cluster, the process should be able to continue without interruption
1. If the user who started a long-lived process is removed from access to the cluster, the process should be able to continue without interruption
2. If the user who started processes are removed from the cluster, administrators may wish to terminate their processes in bulk
2. If the user who started processes are removed from the cluster, administrators may wish to terminate their processes in bulk
3. When containers run with a service account, the user that created / triggered the service account behavior must be associated with the container's action
3. When containers run with a service account, the user that created / triggered the service account behavior must be associated with the container's action
5. When container processes run on the cluster, they should run in a [security context](./security_context.md) that isolates those processes via Linux user security, user namespaces, and permissions.
5. When container processes run on the cluster, they should run in a [security context](security_context.md) that isolates those processes via Linux user security, user namespaces, and permissions.
1. Administrators should be able to configure the cluster to automatically confine all container processes as a non-root, randomly assigned UID
1. Administrators should be able to configure the cluster to automatically confine all container processes as a non-root, randomly assigned UID
2. Administrators should be able to ensure that container processes within the same namespace are all assigned the same unix user UID
2. Administrators should be able to ensure that container processes within the same namespace are all assigned the same unix user UID
3. Administrators should be able to limit which developers and project administrators have access to higher privilege actions
3. Administrators should be able to limit which developers and project administrators have access to higher privilege actions
...
@@ -79,7 +79,7 @@ A pod runs in a *security context* under a *service account* that is defined by
...
@@ -79,7 +79,7 @@ A pod runs in a *security context* under a *service account* that is defined by
6. Developers may need to ensure their images work within higher security requirements specified by administrators
6. Developers may need to ensure their images work within higher security requirements specified by administrators
7. When available, Linux kernel user namespaces can be used to ensure 5.2 and 5.4 are met.
7. When available, Linux kernel user namespaces can be used to ensure 5.2 and 5.4 are met.
8. When application developers want to share filesytem data via distributed filesystems, the Unix user ids on those filesystems must be consistent across different container processes
8. When application developers want to share filesytem data via distributed filesystems, the Unix user ids on those filesystems must be consistent across different container processes
6. Developers should be able to define [secrets](./secrets.md) that are automatically added to the containers when pods are run
6. Developers should be able to define [secrets](secrets.md) that are automatically added to the containers when pods are run
1. Secrets are files injected into the container whose values should not be displayed within a pod. Examples:
1. Secrets are files injected into the container whose values should not be displayed within a pod. Examples:
1. An SSH private key for git cloning remote data
1. An SSH private key for git cloning remote data
2. A client certificate for accessing a remote system
2. A client certificate for accessing a remote system
...
@@ -93,11 +93,11 @@ A pod runs in a *security context* under a *service account* that is defined by
...
@@ -93,11 +93,11 @@ A pod runs in a *security context* under a *service account* that is defined by
### Related design discussion
### Related design discussion
*[Authorization and authentication](./access.md)
*[Authorization and authentication](access.md)
*[Secret distribution via files](https://github.com/GoogleCloudPlatform/kubernetes/pull/2030)
*[Secret distribution via files](https://github.com/GoogleCloudPlatform/kubernetes/pull/2030)
@@ -9,7 +9,7 @@ Running kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/deve
...
@@ -9,7 +9,7 @@ Running kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/deve
2.[VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware)
2.[VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware)
3.[VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware)
3.[VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware)
4.[Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/)
4.[Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/)
3. Get or build a [binary release](/docs/getting-started-guides/binary_release.md)
3. Get or build a [binary release](../../../docs/getting-started-guides/binary_release.md)
### Setup
### Setup
...
@@ -244,7 +244,7 @@ my-nginx nginx run=my-nginx 3
...
@@ -244,7 +244,7 @@ my-nginx nginx run=my-nginx 3
```
```
We did not start any services, hence there are none listed. But we see three replicas displayed properly.
We did not start any services, hence there are none listed. But we see three replicas displayed properly.
Check the [guestbook](/examples/guestbook/README.md) application to learn how to create a service.
Check the [guestbook](../../../examples/guestbook/README.md) application to learn how to create a service.
You can already play with scaling the replicas with:
You can already play with scaling the replicas with:
@@ -15,7 +15,7 @@ CloudStack is a software to build public and private clouds based on hardware vi
...
@@ -15,7 +15,7 @@ CloudStack is a software to build public and private clouds based on hardware vi
[CoreOS](http://coreos.com) templates for CloudStack are built [nightly](http://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions.
[CoreOS](http://coreos.com) templates for CloudStack are built [nightly](http://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions.
This guide uses an [Ansible playbook](https://github.com/runseb/ansible-kubernetes).
This guide uses an [Ansible playbook](https://github.com/runseb/ansible-kubernetes).
This is a completely automated, a single playbook deploys Kubernetes based on the coreOS [instructions](./coreos/coreos_multinode_cluster.md).
This is a completely automated, a single playbook deploys Kubernetes based on the coreOS [instructions](coreos/coreos_multinode_cluster.md).
This [Ansible](http://ansibleworks.com) playbook deploys Kubernetes on a CloudStack based Cloud using CoreOS images. The playbook, creates an ssh key pair, creates a security group and associated rules and finally starts coreOS instances configured via cloud-init.
This [Ansible](http://ansibleworks.com) playbook deploys Kubernetes on a CloudStack based Cloud using CoreOS images. The playbook, creates an ssh key pair, creates a security group and associated rules and finally starts coreOS instances configured via cloud-init.
@@ -150,7 +150,7 @@ Some of the pods may take a few seconds to start up (during this time they'll sh
...
@@ -150,7 +150,7 @@ Some of the pods may take a few seconds to start up (during this time they'll sh
Then, see [a simple nginx example](../../examples/simple-nginx.md) to try out your new cluster.
Then, see [a simple nginx example](../../examples/simple-nginx.md) to try out your new cluster.
For more complete applications, please look in the [examples directory](../../examples). The [guestbook example](../../examples/guestbook) is a good "getting started" walkthrough.
For more complete applications, please look in the [examples directory](../../examples/). The [guestbook example](../../examples/guestbook/) is a good "getting started" walkthrough.
### Tearing down the cluster
### Tearing down the cluster
To remove/delete/teardown the cluster, use the `kube-down.sh` script.
To remove/delete/teardown the cluster, use the `kube-down.sh` script.
This diagram shows four nodes created on a Google Compute Engine cluster with the name of each VM node on a purple background. The internal and public IPs of each node are shown on gray boxes and the pods running in each node are shown in green boxes. Each pod box shows the name of the pod and the namespace it runs in, the IP address of the pod and the images which are run as part of the pod’s execution. Here we see that every node is running a fluentd-cloud-logging pod which is collecting the log output of the containers running on the same node and sending them to Google Cloud Logging. A pod which provides the
This diagram shows four nodes created on a Google Compute Engine cluster with the name of each VM node on a purple background. The internal and public IPs of each node are shown on gray boxes and the pods running in each node are shown in green boxes. Each pod box shows the name of the pod and the namespace it runs in, the IP address of the pod and the images which are run as part of the pod’s execution. Here we see that every node is running a fluentd-cloud-logging pod which is collecting the log output of the containers running on the same node and sending them to Google Cloud Logging. A pod which provides the
[cluster DNS service](/docs/dns.md) runs on one of the nodes and a pod which provides monitoring support runs on another node.
[cluster DNS service](../../docs/dns.md) runs on one of the nodes and a pod which provides monitoring support runs on another node.
To help explain how cluster level logging works let’s start off with a synthetic log generator pod specification [counter-pod.yaml](/examples/blog-logging/counter-pod.yaml):
To help explain how cluster level logging works let’s start off with a synthetic log generator pod specification [counter-pod.yaml](../../examples/blog-logging/counter-pod.yaml):
```
```
apiVersion: v1
apiVersion: v1
kind: Pod
kind: Pod
...
@@ -55,7 +55,7 @@ This step may take a few minutes to download the ubuntu:14.04 image during which
...
@@ -55,7 +55,7 @@ This step may take a few minutes to download the ubuntu:14.04 image during which
When the pod status changes to `Running` we can use the kubectl logs command to view the output of this counter pod.
When the pod status changes to `Running` we can use the kubectl logs command to view the output of this counter pod.
...
@@ -114,7 +114,7 @@ We’ve lost the log lines from the first invocation of the container in this po
...
@@ -114,7 +114,7 @@ We’ve lost the log lines from the first invocation of the container in this po
When a Kubernetes cluster is created with logging to Google Cloud Logging enabled, the system creates a pod called `fluentd-cloud-logging` on each node of the cluster to collect Docker container logs. These pods were shown at the start of this blog article in the response to the first get pods command.
When a Kubernetes cluster is created with logging to Google Cloud Logging enabled, the system creates a pod called `fluentd-cloud-logging` on each node of the cluster to collect Docker container logs. These pods were shown at the start of this blog article in the response to the first get pods command.
This log collection pod has a specification which looks something like this [fluentd-gcp.yaml](/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml):
This log collection pod has a specification which looks something like this [fluentd-gcp.yaml](../../cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml):
```
```
apiVersion: v1
apiVersion: v1
...
@@ -187,7 +187,7 @@ Now we can run queries over the ingested logs. The example below uses the [jq](h
...
@@ -187,7 +187,7 @@ Now we can run queries over the ingested logs. The example below uses the [jq](h
...
...
```
```
This page has touched briefly on the underlying mechanisms that support gathering cluster level logs on a Kubernetes deployment. The approach here only works for gathering the standard output and standard error output of the processes running in the pod’s containers. To gather other logs that are stored in files one can use a sidecar container to gather the required files as described at the page [Collecting log files within containers with Fluentd](/contrib/logging/fluentd-sidecar-gcp/README.md) and sending them to the Google Cloud Logging service.
This page has touched briefly on the underlying mechanisms that support gathering cluster level logs on a Kubernetes deployment. The approach here only works for gathering the standard output and standard error output of the processes running in the pod’s containers. To gather other logs that are stored in files one can use a sidecar container to gather the required files as described at the page [Collecting log files within containers with Fluentd](../../contrib/logging/fluentd-sidecar-gcp/README.md) and sending them to the Google Cloud Logging service.
Some of the material in this section also appears in the blog article [Cluster Level Logging with Kubernetes](http://blog.kubernetes.io/2015/06/cluster-level-logging-with-kubernetes.html).
Some of the material in this section also appears in the blog article [Cluster Level Logging with Kubernetes](http://blog.kubernetes.io/2015/06/cluster-level-logging-with-kubernetes.html).
@@ -46,11 +46,11 @@ choices. For example, on systemd-based systems (e.g. RHEL, CentOS), you can run
...
@@ -46,11 +46,11 @@ choices. For example, on systemd-based systems (e.g. RHEL, CentOS), you can run
If you are extending from a standard Kubernetes installation, the ```kubelet``` binary should already be present on your system. You can run
If you are extending from a standard Kubernetes installation, the ```kubelet``` binary should already be present on your system. You can run
```which kubelet``` to determine if the binary is in fact installed. If it is not installed,
```which kubelet``` to determine if the binary is in fact installed. If it is not installed,
you should install the [kubelet binary](https://storage.googleapis.com/kubernetes-release/release/v0.19.3/bin/linux/amd64/kubelet), the
you should install the [kubelet binary](https://storage.googleapis.com/kubernetes-release/release/v0.19.3/bin/linux/amd64/kubelet), the
[/etc/init.d/kubelet](high-availability/init-kubelet) and [/etc/default/kubelet](high-availability/default-kubelet)
[high-availability/init-kubelet](high-availability/init-kubelet) and [high-availability/default-kubelet](high-availability/default-kubelet)
scripts.
scripts.
If you are using monit, you should also install the monit daemon (```apt-get install monit```) and the [/etc/monit/conf.d/kubelet](high-availability/monit-kubelet) and
If you are using monit, you should also install the monit daemon (```apt-get install monit```) and the [high-availability/monit-kubelet](high-availability/monit-kubelet) and
@@ -6,7 +6,7 @@ Kubernetes components, such as kubelet and apiserver, use the [glog](https://god
...
@@ -6,7 +6,7 @@ Kubernetes components, such as kubelet and apiserver, use the [glog](https://god
## Examining the logs of running containers
## Examining the logs of running containers
The logs of a running container may be fetched using the command `kubectl logs`. For example, given
The logs of a running container may be fetched using the command `kubectl logs`. For example, given
this pod specification which has a container which writes out some text to standard
this pod specification which has a container which writes out some text to standard
output every second [counter-pod.yaml](/examples/blog-logging/counter-pod.yaml):
output every second [counter-pod.yaml](../examples/blog-logging/counter-pod.yaml):
```
```
apiVersion: v1
apiVersion: v1
kind: Pod
kind: Pod
...
@@ -66,6 +66,6 @@ describes how to ingest cluster level logs into Elasticsearch and view them usin
...
@@ -66,6 +66,6 @@ describes how to ingest cluster level logs into Elasticsearch and view them usin
## Ingesting Application Log Files
## Ingesting Application Log Files
Cluster level logging only collects the standard output and standard error output of the applications
Cluster level logging only collects the standard output and standard error output of the applications
running in containers. The guide [Collecting log files within containers with Fluentd](/contrib/logging/fluentd-sidecar-gcp/README.md) explains how the log files of applications can also be ingested into Google Cloud logging.
running in containers. The guide [Collecting log files within containers with Fluentd](../contrib/logging/fluentd-sidecar-gcp/README.md) explains how the log files of applications can also be ingested into Google Cloud logging.
This document describes the current state of `PersistentVolumes` in Kubernetes. Familiarity with [volumes](./volumes.md) is suggested.
This document describes the current state of `PersistentVolumes` in Kubernetes. Familiarity with [volumes](volumes.md) is suggested.
Managing storage is a distinct problem from managing compute. The `PersistentVolume` subsystem provides an API for users and administrators that abstracts details of how storage is provided from how it is consumed. To do this we introduce two new API resources: `PersistentVolume` and `PersistentVolumeClaim`.
Managing storage is a distinct problem from managing compute. The `PersistentVolume` subsystem provides an API for users and administrators that abstracts details of how storage is provided from how it is consumed. To do this we introduce two new API resources: `PersistentVolume` and `PersistentVolumeClaim`.
...
@@ -8,7 +8,7 @@ A `PersistentVolume` (PV) is a piece of networked storage in the cluster that ha
...
@@ -8,7 +8,7 @@ A `PersistentVolume` (PV) is a piece of networked storage in the cluster that ha
A `PersistentVolumeClaim` (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g, can be mounted once read/write or many times read-only).
A `PersistentVolumeClaim` (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g, can be mounted once read/write or many times read-only).
Please see the [detailed walkthrough with working examples](../examples/persistent-volumes).
Please see the [detailed walkthrough with working examples](../examples/persistent-volumes/).
## Lifecycle of a volume and claim
## Lifecycle of a volume and claim
...
@@ -77,7 +77,7 @@ Each PV contains a spec and status, which is the specification and status of the
...
@@ -77,7 +77,7 @@ Each PV contains a spec and status, which is the specification and status of the
### Capacity
### Capacity
Generally, a PV will have a specific storage capacity. This is set using the PV's `capacity` attribute. See the Kubernetes [Resource Model](./design/resources.md) to understand the units expected by `capacity`.
Generally, a PV will have a specific storage capacity. This is set using the PV's `capacity` attribute. See the Kubernetes [Resource Model](design/resources.md) to understand the units expected by `capacity`.
Currently, storage size is the only resource that can be set or requested. Future attributes may include IOPS, throughput, etc.
Currently, storage size is the only resource that can be set or requested. Future attributes may include IOPS, throughput, etc.
...
@@ -145,7 +145,7 @@ Claims use the same conventions as volumes when requesting storage with specific
...
@@ -145,7 +145,7 @@ Claims use the same conventions as volumes when requesting storage with specific
### Resources
### Resources
Claims, like pods, can request specific quantities of a resource. In this case, the request is for storage. The same [resource model](./design/resources.md) applies to both volumes and claims.
Claims, like pods, can request specific quantities of a resource. In this case, the request is for storage. The same [resource model](design/resources.md) applies to both volumes and claims.
## <a name="claims-as-volumes"></a> Claims As Volumes
## <a name="claims-as-volumes"></a> Claims As Volumes