> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Long-term downsampling

> Learn how long-term downsampling compacts persisted metrics and its effect on each metric type.

After approximately five days, each persisted metric, including both raw and
aggregated metrics, undergoes a process called *long-term downsampling*. This
process temporally downsamples data into a more compact form, and then deletes the
non-downsampled data permanently.

To maintain an accurate representation of the data, Palo Alto Networks Cortex XCOR uses different
downsampling methodologies, depending on the metric type.

<Note>
  These behaviors are important to note beforehand, since any unexpected results of
  long-term downsampling will be noticed only approximately five days after ingestion.
</Note>

By default, Cortex XCOR downsamples long term data at a five-minute
granularity, where all data points within each five-minute window compress into a
single data point. This five-minute window is termed the *downsample window*.

## Effects on metric types

Downsampling effects differ based on the [metric type](/control/shaping/shape-metrics/types).

* **Cumulative counter**:
  [Cumulative counters](/control/shaping/shape-metrics/types#cumulative-counter) downsample by
  preserving the overall increase (respecting resets) between the start and end of
  the downsample window. This reduces the temporal granularity by observing only
  one increase every five minutes, while keeping the running count accurate.

* **Delta counters**: [Delta counters](/control/shaping/shape-metrics/types#delta-counter)
  downsample by preserving the overall increase between the start
  and end of the downsample window. The `sum_over_time()` should remain the same,
  but at a reduced granularity, such as every five minutes.

* **Gauge**: Downsampling of [gauges](/control/shaping/shape-metrics/types#gauge) differs,
  depending on how the gauge was ingested or if the gauge originated from a rollup rule.

  By default, gauges downsample by preserving only the last data point in every
  downsample window. Any changes to the gauge prior to the end of the downsample window
  aren't retained.

  If the gauge is an output of a
  [`MIN/MAX` aggregation](/control/shaping/shape-metrics/rules/rollup#gauge), the gauge is
  downsampled by preserving the `MIN/MAX` data point in every downsample window,
  respectively.

  Gauges ingested with StatsD downsample using a Largest-Triangle-Three-Buckets
  (LTTB) downsampling algorithm for consistency with the Graphite query engine.
  Graphite statistics use a sum downsampling for counters, and LTTB for timers and gauges.

* **Histograms**: Cumulative exponential histograms are downsampled by preserving the
  overall increase (respecting resets) of each bucket count between the start and
  end of the downsample window. Delta exponential histograms are downsampled by
  summing the data points in the downsampling window.

  Due to the reduced temporal granularity, you'll see changes no more frequently
  than every 5 minutes, while the running bucket counts remain accurate.

  If the histogram exceeds the 160-bucket limit, Cortex XCOR decreases
  its scale until the bucket count is within the limit. Downscaling reduces the
  histogram's resolution.

  Classic Prometheus histograms are cumulative counters and have the same downsampling
  effects.

## Querying downsampled data

Querying downsampled data can produce different results, especially between higher
and lower resolution data. Some queries can be sensitive to data resolution. Although
a query might work well with raw data, it could return no data, or data that doesn't
make sense after downsampling.

For example, `rate(some_metric[2m])` with a raw resolution of `30s` returns
results. If you increase the query time range, the query requests downsampled data.
The results are empty, because the query requested multiple data points in the `2m`
window, but downsampling reduces available data to one data point every
[five minutes](/administer/limits-licensing/licensing#contract-details).

To prevent empty graphs when changing the query time range, Cortex XCOR
enforces a minimum range selector on queries that read downsampled data. The minimum
is three times the resolution of the downsampled data, which is `15m` at the default
`5m` resolution. Cortex XCOR rewrites any range selector shorter than the
minimum to the minimum, and leaves longer range selectors unchanged.

A range selector that equals or doubles the resolution is still rewritten, because
both remain shorter than three times the resolution. The following table shows how
Cortex XCOR rewrites the previous example at a `5m` resolution:

| Original query | Query that runs | Reason |
| - | - | - |
| `rate(some_metric[2m])` | `rate(some_metric[15m])` | `2m` is shorter than the `15m` minimum. |
| `rate(some_metric[5m])` | `rate(some_metric[15m])` | `5m` matches the resolution, but is shorter than the minimum. |
| `rate(some_metric[10m])` | `rate(some_metric[15m])` | `10m` is twice the resolution, but shorter than the minimum. |
| `rate(some_metric[15m])` | `rate(some_metric[15m])` | `15m` equals the minimum, so no rewrite occurs. |
| `rate(some_metric[30m])` | `rate(some_metric[30m])` | `30m` exceeds the minimum, so no rewrite occurs. |

Querying lower resolution data might also affect some of the function output,
including:

* The `increase()` function can return very different numbers after switching to
  downsampled data.
* The `rate()` function smooths out peaks for graphs with many peaks with raw data.
* Any `resets()` are dropped when downsampling data.


## Related topics

- [Metric types](/control/shaping/shape-metrics/types.md)
- [Troubleshooting missing metrics](/investigate/querying/metrics/troubleshooting.md)
- [Shape metric data](/control/shaping/shape-metrics.md)
- [Using custom PromQL functions in Cortex XCOR](/investigate/querying/promql/custom-functions.md)
- [Differences between the Prometheus Operator and Chronosphere Collector](/ingest/metrics-traces/collector/mappings/prometheus/operator-collector.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.