> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Manage metric pools
> Add, edit, and preview metric pool quota allocations on the Metrics Quotas page in Palo Alto Networks Cortex XCOR.
On the [Metrics Quotas](/control/shaping/shape-metrics/quotas/quotas-ui) page in
Palo Alto Networks Cortex XCOR, review current quota allocations and preview
potential changes by pool. Use this feature to plan pool sizes and future usage.
To get started with pools, learn about how to
[define a pool](/control/shaping/shape-metrics/quotas/define-pools).
## Add a metric pool
Select from the following methods to add a metric pool. Changes made in the web
interface are previews that you apply using the generated configuration.
You can define up to 20 custom metric pools.
Although actual management of pools is handled using
[Terraform](/tooling/infrastructure/terraform), this interface helps you
understand what changes to make to reduce guesswork and repeated updates to your
system.
To create quota pools, you must have administrative privileges:
1. In the navigation menu, click ** Go to Admin**
and then select
** Optimization > Metrics Quotas**.
2. Click **Configure Quotas**.
3. Click **+ Add Pool**.
4. The following fields display on the page. Update editable fields to modify your
pool configuration:
* **Pool name:** (Editable) Change the pool name.
* **Data matching:** The values the selected pool uses to match data.
* **Quota configuration label:** The label matching this pool.
* **Data matching values:** (Editable) The specific values for the label, which
match this pool. Add a value and press `Enter` or Space to add it to the list.
The value supports [glob syntax](/investigate/querying/glob-syntax).
* **Observed label consumption:** Displays estimated Standard Metrics consumption
for the matching label values, broken down by label value. The estimate isn't
restricted to data currently assigned to the pool.
* **Quota allocation:** (Editable) Set a quota percentage for the selected pool.
The drawer displays the equivalent persisted writes Standard Metrics value in
data points per second (DPPS). To enter a DPPS value, use the **DPPS** mode in
the **Pools** table.
* **Prioritization:** (Conditionally editable) Add the **Priority Label** and high or
low priority values.
Pools using a global priority setting can't change their priorities for an
individual pool.
5. In the **Quota Allocation** chart, select or clear pool names in the legend to
include or remove their segments from the **Allocation** and
**Current Consumption** stacked bars.
6. Click **Done** after completing your changes.
7. Click the **Code config** tab.
8. Click ** Copy** to copy the file, or
** Download** to download the file to your computer.
9. Review the generated configuration. Percentage changes apply to licenses that
don't have fixed-value overrides. Changes entered in **DPPS** mode update only the
`PERSISTED_WRITES_STANDARD` fixed value. The generated configuration preserves
existing fixed values and thresholds for other licenses.
10. Add the definition to a Terraform file, or create a new Terraform file.
11. Run this command to apply the resource:
```shell theme={null}
terraform apply
```
To create a pool using [Chronoctl](/tooling/chronoctl):
1. Run the following command to generate a sample pool configuration you can use as a
template:
```shell theme={null}
chronoctl resource-pools scaffold
```
In the template, `kind: ResourcePools` defines the complete pool configuration.
The `ResourcePools` resource is a singleton that contains the default pool and all
named pools. If a `ResourcePools` resource already exists, follow the
[Chronoctl edit procedure](#edit-a-pool) instead of creating another resource.
2. With a completed definition, submit it with:
```shell theme={null}
chronoctl resource-pools create -f FILE_NAME.yaml
```
Replace *`FILE_NAME`* with the name of the YAML definition file you want to use.
When you run `terraform plan` to generate an execution plan, Cortex XCOR automatically
tests configurations that include notification policies by submitting them as dry runs.
For details, see the
[Terraform provider](/tooling/infrastructure/terraform#validate-plans-with-dry-runs)
documentation.
To create a pool with [Terraform](/tooling/infrastructure/terraform):
1. Create or edit a Terraform file and add the definition by using the
`chronosphere_resource_pools_config` type, followed by a name in a resource
declaration.
2. Run this command to apply the changes:
```shell theme={null}
terraform apply
```
See the [Terraform pool example](#terraform-pool-example) for a completed
pool resource.
To complete this action with the Cortex XCOR API, use the
[`CreateResourcePools`](/tooling/api-info/definition/operations/CreateResourcePools)
endpoint.
The endpoint creates the complete `ResourcePools` singleton. If the resource already
exists, use the
[`UpdateResourcePools`](/tooling/api-info/definition/operations/UpdateResourcePools)
endpoint to add a named pool.
Because the Cortex XCOR API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
### Chronoctl pool example
The following code is an example of a Chronoctl resource used to create quotas and
priorities.
The example creates two pools: one for `team_a` and one for `team_b`. Each pool
defines license values for persisted writes and matched writes, along with
[thresholds](/control/shaping/shape-metrics/quotas/define-pools#pool-thresholds) to
manage persisted cardinality.
```yaml expandable Chronoctl example icon="square-terminal" theme={null}
api_version: v1/config
kind: ResourcePools
spec:
pools:
- name: team_a
allocation:
fixed_values:
- license: PERSISTED_WRITES_STANDARD
value: "70000"
- license: MATCHED_WRITES_STANDARD
value: "30000"
priority_thresholds:
- license: PERSISTED_CARDINALITY_STANDARD
all_priorities:
fixed_value: "6000000"
low_priority:
fixed_value: "2000000"
filters:
- name: team
value_glob: 'a'
priorities:
high_priority_filters:
- name: env
value_glob: '*prod*'
low_priority_filters:
- name: env
value_glob: '*dev*'
- name: team_b
allocation:
fixed_values:
- license: PERSISTED_WRITES_STANDARD
value: "16000"
- license: MATCHED_WRITES_STANDARD
value: "15000"
priority_thresholds:
- license: PERSISTED_CARDINALITY_STANDARD
all_priorities:
fixed_value: "1750000"
low_priority:
fixed_value: "750000"
filters:
- name: team
value_glob: 'b'
priorities:
high_priority_filters:
- name: env
value_glob: '*prod*'
low_priority_filters:
- name: env
value_glob: '*dev*'
```
### Terraform pool example
The following code is an example of a Terraform file used to create pools and
priorities.
This code is an example and shouldn't be used directly. This example uses the
following variables in place of a specific name:
* *`MY_RESOURCE`* is a resource in your system.
* *`MY_SERVICE`* is the name of a service in your system.
```terraform expandable Terraform example icon="square-terminal" theme={null}
resource "MY_RESOURCE_resource_pools_config" "resource_pools" {
default_pool {
priorities {
high_priority_match_rules = ["MY_RESOURCE_k8s_cluster:production*"]
low_priority_match_rules = ["MY_RESOURCE_k8s_cluster:rc*"]
}
}
pool {
name = "Tracing Services"
allocation {
# Applies to each license that doesn't have a fixed-value override.
percent_of_license = 10
}
match_rules = ["service:{spanhandler,traceingester}"]
priorities {
# Optional. Match rules are filters that define which metrics are high or
# low priority. Any metric that matches at least one filter is considered
# high or low priority, depending on the defined priority. High priority
# match rules take precedence over low priority ones. If a metric
# matches both a high and low priority rule, it's considered a high
# priority metric. When the license limit is exceeded, high priority
# metrics are dropped last, and low priority metrics are dropped first.
# This behavior applies to only persisted writes and matched writes, but
# doesn't apply to persisted cardinality.
high_priority_match_rules = ["MY_RESOURCE_k8s_cluster:production*"]
low_priority_match_rules = ["MY_RESOURCE_k8s_cluster:rc*"]
}
}
pool {
name = "Collector service"
allocation {
percent_of_license = 8
# Optional. For supported licenses, defines thresholds with strict limits
# for when to drop new consumption of the license for a pool. Currently, only
# `PERSISTED_CARDINALITY_STANDARD` and `PERSISTED_CARDINALITY_HISTOGRAM` are
# supported.
priority_thresholds {
license = "PERSISTED_CARDINALITY_STANDARD"
# Threshold limit that defines when to drop new metrics in the pool. This
# threshold applies to all priorities of metrics: high, default, and low.
# This field must be set, and have a value equal to or greater than other
# priority fields.
all_priorities {
percent_of_pool_allocation = 100
}
}
priority_thresholds {
license = "PERSISTED_CARDINALITY_HISTOGRAM"
all_priorities {
percent_of_pool_allocation = 100
}
}
}
match_rules = ["service:{${join(",", local.collector_services)}}"]
priorities {
high_priority_match_rules = ["chronosphere_k8s_cluster:prod*"]
low_priority_match_rules = ["chronosphere_k8s_cluster:rc*"]
}
}
pool {
name = "MY_SERVICE Services"
allocation {
percent_of_license = 25
}
match_rules = ["service:MY_SERVICE*"]
priorities {
high_priority_match_rules = ["MY_RESOURCE_k8s_cluster:production*"]
low_priority_match_rules = ["MY_RESOURCE_k8s_cluster:rc*"]
}
}
pool {
name = "Gateway Services"
allocation {
percent_of_license = 4
}
match_rules = ["service:gateway*"]
priorities {
high_priority_match_rules = ["MY_RESOURCE_k8s_cluster:production*"]
low_priority_match_rules = ["MY_RESOURCE_k8s_cluster:rc*"]
}
}
}
```
## Edit a pool
Select from the following methods to edit pools. You can also
[configure global priorities](/control/shaping/shape-metrics/quotas/define-pools#configure-global-priority)
to change global pool quota configurations by metric label.
The visual editor requires one quota configuration label across all pools and one
prioritization label within each pool. Edit configurations that use multiple matching
or prioritization labels directly in Terraform.
To edit an existing pool:
1. In the navigation menu, click ** Go to Admin**
and then select
** Optimization > Metrics Quotas**.
2. Click **Configure Quotas**.
3. Click any row in the **Pools** table to open the **Edit Pool** drawer.
The **Edit Pool** drawer contains information specific to the selected pool. These
fields match the **Add Pool** screen, and some values can be edited.
4. Make any necessary changes, and then click **Done** after completing your changes.
5. Click the **Code config** tab.
6. Click ** Copy** to copy the file, or
** Download** to download the file to your computer.
7. Review the generated configuration to confirm that existing fixed values and
priority thresholds are preserved.
8. Add the definition to a Terraform file, or create a new Terraform file.
9. Run this command to apply the resource:
```shell theme={null}
terraform apply
```
To edit a pool using [Chronoctl](/tooling/chronoctl):
1. View your existing metrics pools YAML definition with the
`chronoctl resource-pools read` command:
```shell theme={null}
chronoctl resource-pools read
```
2. Copy the YAML definition and save it to a new file ending in `.yaml`.
3. Modify the YAML definition properties and apply the changes:
```shell theme={null}
chronoctl resource-pools update -f FILE_NAME.yaml
```
Replace *`FILE_NAME`* with the name of the YAML definition file you want to use.
To edit a pool using [Terraform](/tooling/infrastructure/terraform):
1. Create or edit a Terraform file that updates the resource's existing properties.
2. Run this command to apply the changes:
```shell theme={null}
terraform apply
```
You can also use the Code config tool to view and edit the resource pool Terraform
representation:
1. Click the **Code config** tab.
2. Make changes to the resource pool definition.
3. Click ** Copy** to copy the file, or
** Download** to download the file to your computer.
4. Add the definition to a Terraform file, or create a new Terraform file.
5. Run this command to apply the resource:
```shell theme={null}
terraform apply
```
To complete this action with the Cortex XCOR API, use the
[`UpdateResourcePools`](/tooling/api-info/definition/operations/UpdateResourcePools) endpoint.
Because the Cortex XCOR API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
### Understanding pool usage
The **Pools** section of the **Configure Quotas** page describes what pools you have,
and how they're configured. This includes:
* **Pool name:** The display name of the pool.
* **Data matching:** The label values the pool matches.
* **Allocation:** The percentage or DPPS of total traffic guaranteed to the pool
before it might be penalized.
* **Consumption:** The percentage or DPPS of total traffic the pool is consuming over
the selected [time range](/navigate/time-ranges).
Allocation and consumption DPPS values describe
[Standard Metrics License](/administer/limits-licensing/concepts/capacity-licensing#standard-metrics-license)
allocation and consumption.
Histogram Metrics License allocation and consumption aren't included in the pool's
reported DPPS.
#### Quota allocations and consumption
The **Quota Allocations vs Current Consumption** graph compares two stacked bars.
The **Allocation** bar segments the current quota allocation by pool. The
**Current Consumption** bar segments current consumption by pool. Select pool names
in the legend to filter both bars.
#### Quota consumption by pools (per second)
The **Quota Consumption by pools** graph compares each pool's displayed consumption
with its current quota limit. Changing an allocation adds the
**New Quota Limit** series.
#### Understand quota consumption trends
The **Quota Consumption** graph plots each pool's consumption over time. Point to a
data point to display its value. Drag across the graph to focus on a time period.
#### Preview quota allocations
The **Pools** include a group of text fields corresponding to each created pool. These
boxes contain values with the assigned percentage (**%**) or DPPS for each pool.
Use these to set your general pool allocations.
To preview a new quota allocation, change a number in the box for the pool to be
updated. Click outside the boxes to update the total.
Quota settings must meet the following criteria:
* In the web interface, changing a named pool's quota adjusts the default pool so
that all allocations total 100%.
* The **Add Pool** and **Edit Pool** drawers support values greater than or equal to
0.01%. Allocation fields in the **Pools** table support positive values with up to
three decimal places.
* The API supports values from 0% through 100% with up to three decimal places. If
the default pool allocation is omitted, named pool allocations can total less than
100%, and the default pool receives the remainder.
Changing an assigned quota displays a third bar in the **Quota consumption by pools**
chart. Use the new bar to determine if new quota assignments meet the needs of each
of your pools.
If one pool is consistently over quota and the other pools aren't, use the preview to
adjust assigned quotas to better meet the needs of each pool.
Click the ** Reset quotas** icon to restore existing allocation
values. This action doesn't undo changes to names, data matching, priorities, added
pools, or removed pools.
## Delete a pool
Select from the following methods to delete pools.
To delete an existing pool:
1. In the navigation menu, click ** Go to Admin**
and then select
** Optimization > Metrics Quotas**.
2. Click **Configure Quotas**.
3. Click a non-default pool's row in the **Pools** table to open the
**Edit Pool** drawer.
4. At the bottom of the **Edit Pool** drawer, click **Remove Pool**.
The pool is removed from the preview and the resource definition in the
**Code config** tab.
5. Click the **Code config** tab.
6. Click ** Copy** to copy the file, or
** Download** to download the file to your computer.
7. Update your Terraform file with the generated definition.
8. Run this command to apply the change:
```shell theme={null}
terraform apply
```
To delete a pool using [Chronoctl](/tooling/chronoctl):
1. View your existing metrics pools YAML definition with the
`chronoctl resource-pools read` command:
```shell theme={null}
chronoctl resource-pools read
```
2. Copy the YAML definition and save it to a new file ending in `.yaml`.
3. Remove the pool you want to delete from the YAML definition and save the updated
file.
4. Apply the changes:
```shell theme={null}
chronoctl resource-pools update -f FILE_NAME.yaml
```
Replace *`FILE_NAME`* with the name of the YAML definition file you want to use.
To delete the *entire* `ResourcePools` definition, use the
`chronoctl resource-pools delete` command:
```shell theme={null}
chronoctl resource-pools delete
```
This command deletes the *entire* `ResourcePools` definition.
To delete a resource that's managed by [Terraform](/tooling/infrastructure/terraform):
1. Edit your Terraform configuration file to remove the pre-existing resource
definition.
2. Run this command to remove the resource from Cortex XCOR:
```shell theme={null}
terraform apply
```
You can also use the Code config tool to view and edit the resource pool Terraform
representation:
1. Click the **Code config** tab.
2. Make changes to the resource pool definition.
3. Click ** Copy** to copy the file, or
** Download** to download the file to your computer.
4. Add the definition to a Terraform file, or create a new Terraform file.
5. Run this command to apply the resource:
```shell theme={null}
terraform apply
```
To complete this action with the Cortex XCOR API, use the
[`DeleteResourcePools`](/tooling/api-info/definition/operations/DeleteResourcePools)
endpoint.
Because the Cortex XCOR API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
This action deletes the *entire* `ResourcePools` definition.
## Best practices
To keep penalty behavior and cost accounting transparent and predictable, pools
should be hard partitions of your system, with no one time series matching
more than one pool. The following processes help ensure pools have the correct data:
* Cortex XCOR recommends selecting a single usage tag as the pool assignment
mechanism. Picking a single tag reduces the possibility where one pool matches
`serviceX`, and a second pool matches `environmentY`, where time series might match
either or both definitions.
* Use exact match values for the selected label. The Cortex XCOR API rejects an exact label value
that's already assigned to another pool.
* Cortex XCOR uses match ordering for valid glob patterns that overlap. If
a time series matches more than one pool, it becomes part of the first matching pool
in the list.
* If you see a pool that doesn't match the expected penalty behavior, open the pool
in the profiler and compare it with the Terraform configuration file. A match rule
value might be incorrect.
## Create an alert for a pool in penalty
When a pool is in a penalty state, it might drop metrics to reduce usage. For higher
priority pools, this can result in the loss of important data. To reduce or prevent
data loss, create a [monitor](/investigate/alerts/monitors) to alert on pool usage and
[send notifications](/investigate/alerts/notifications) to the appropriate team.
1. [Create a notifier](/investigate/alerts/notifications/notifiers) and align with
your internal alerting policies to route the alerts to the right team.
2. Create a [notification policy](/investigate/alerts/notifications/policies) that
connects the notifier to a monitor.
3. [Create a monitor](/investigate/alerts/monitors#create-a-monitor).
4. In the monitor query, add the query to return each pool's percent utilization. For
example, the following query calculates persisted writes standard metrics
utilization and excludes pools with zero allocation:
```text wrap theme={null}
(
100 *
sum by (pool_name) (
chrono_metrics_persisted_writes_license_dpps_consumed{datapoint_type="standard"}
)
/
sum by (pool_name) (
chrono_metrics_persisted_writes_license_dpps_capacity{datapoint_type="standard"}
)
)
and on (pool_name)
(
sum by (pool_name) (
chrono_metrics_persisted_writes_license_dpps_capacity{datapoint_type="standard"}
) > 0
)
```
5. In the **Signals** section, select **Per time series (many alerts)** as the
[signal](/investigate/alerts/notifications/signals#per-time-series).
6. Define the warning condition as greater than 90 and set the sustain period to
5 minutes.
7. Click **Save** to save the monitor.
## Related topics
- [Define a metric pool](/control/shaping/shape-metrics/quotas/define-pools.md)
- [Understand metric quotas and pools](/control/shaping/shape-metrics/quotas.md)
- [Cortex XCOR-managed dashboards](/observe/dashboards/managed-dashboards.md)
- [View allocated pools](/control/shaping/shape-metrics/quotas/quotas-ui.md)
- [CreateResourcePools](/tooling/api-info/definition/operations/CreateResourcePools.md)
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.