> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Scaling and replicas

> Learn how Palo Alto Networks Telemetry Pipeline uses scaling and replicas.

<Info>
  This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone
  product separate from Palo Alto Networks Cortex XCOR.
</Info>

<Note>
  The information on this page applies to pipelines with the Deployment or
  StatefulSet [workload type](/ingest/pipeline/v2/configure/kubernetes/workloads), but not
  pipelines with the DaemonSet workload type.
</Note>

Palo Alto Networks Telemetry Pipeline uses replica pods to scale pipelines. Each pipeline
needs at least one replica to function. If a pipeline has more than one replica,
any ingested data is balanced between those replicas accordingly.

You can assign a [fixed number of replicas](#fixed-number-of-replicas) to
a pipeline, or [enable horizontal pod autoscaling](#horizontal-pod-autoscaling)
to automatically add and remove replicas in response to utilization changes.

## Fixed number of replicas

Use one of the following methods to set a fixed number of replicas for your pipelines:

<Note>
  For replica counts with a volume over 100 TB a day, contact
  [Cortex XCOR Support](/support) prior to scaling up to ensure stability.
</Note>

<Tabs>
  <Tab title="Web interface" id="autoscaling-ui">
    1. Sign in to the [Palo Alto Networks Telemetry Pipeline web interface](https://core.calyptia.com/).
    2. Open the project that contains the pipeline that you want to modify.
    3. Go to **Core Instances**, then click the name of the Core Instance associated
       with the pipeline that you want to modify.
    4. Under **Data Pipelines**, click the name of the pipeline that you want to
       modify.
    5. Go to **<Icon icon="square-pen" /> Advanced Settings**.
    6. Increase the **Replicas** field to the amount you want, and then click
       **Apply**.
  </Tab>

  <Tab title="Pipeline CLI" id="autoscaling-cli">
    Use the `update` command to scale an existing pipeline:

    ```shell theme={null}
    calyptia update pipeline PIPELINE_NAME_OR_ID --replicas NUM_OF_REPLICAS
    ```

    To examine the number of replicas for a given pipeline, run:

    ```shell theme={null}
    calyptia get pipelines --core-instance CORE_INSTANCE
    ```

    The output is similar to the following:

    ```text theme={null}
     NAME                 REPLICAS  STATUS   AGE
     pipeline-name               3  STARTED  15 hours
    ```
  </Tab>
</Tabs>

## Horizontal pod autoscaling

> Requires Core Instance version 2.12.3 or later.

Pipelines support [horizontal pod autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)
(HPA), which enables Palo Alto Networks Telemetry Pipeline to add and remove replicas as needed.

### Enable horizontal pod autoscaling

To enable HPA, [create or modify a pipeline](/ingest/pipeline/v2/build/create-modify)
through [Pipeline CLI](/ingest/pipeline/pipeline-cli) and include the
`--min-replicas` and `--max-replicas` flags. You must assign a value greater than
`0` to `--min-replicas`, and the value of `--max-replicas` must be greater than or
equal to the value of `--min-replicas`. For example:

```shell /PIPELINE/ theme={null}
calyptia update pipeline PIPELINE --min-replicas 2 --max-replicas 6
```

Replace *`PIPELINE`* with the name or [unique ID](/ingest/pipeline/v2/configure/metadata#get-pipeline-ids)
of the pipeline you want to modify.

### Configure horizontal pod autoscaling

To configure HPA, use the following flags when you create or update a pipeline through [Pipeline CLI](/ingest/pipeline/pipeline-cli):

| Flag | Description | Value |
| - | - | - |
| `--max-replicas` | The maximum number of replicas to which the autoscaler can scale up. | Example: `6` |
| `--min-replicas` | The minimum number of replicas to which the autoscaler can scale down. | Example: `3` |
| `--scale-up-type` | The type of scale-up policy to implement. If you include this flag, you must also include `--scale-up-value` and `--scale-up-period-seconds`. | `Pods` or `Percent` |
| `--scale-up-value` | The amount of change to implement through scale-up policies. Requires `--scale-up-type`. | Must be greater than `0`. |
| `--scale-up-period-seconds` | The window of time for which the scale-up policy should hold true, in seconds. Requires `--scale-up-type`. | Must be greater than `0`. Example: `30` |
| `--scale-down-type` | The type of scale-down policy to implement. If you include this flag, you must also include `--scale-down-value` and `--scale-down-period-seconds`. | `Pods` or `Percent` |
| `--scale-down-value` | The amount of change to implement through scale-down policies. Requires `--scale-down-type`. | Must be greater than `0`. Example: `15` |
| `--scale-down-period-seconds` | The window of time for which the scale-down policy should hold true, in seconds. Requires `--scale-down-type`. | Must be greater than `0`. Example: `30` |
| `--utilization-cpu-average` | The target percentage value for average CPU utilization. The autoscaler attempts to maintain equilibrium around this value by scaling pipelines up or down accordingly. | Example: `25` |
| `--utilization-memory-average` | The target percentage value for average memory utilization. The autoscaler attempts to maintain equilibrium around this value by scaling pipelines up or down accordingly. | Example: `30` |

Scale-up and scale-down policies are optional. If you set a scale-up or
scale-down policy, you must specify the type, value, and period for that policy in
the same command. For example:

```shell /PIPELINE/ theme={null}
calyptia update pipeline PIPELINE --scale-up-type Pods --scale-up-value 2 --scale-up-period-seconds 30
```

### Disable horizontal pod autoscaling

To disable HPA, update a pipeline through [Pipeline CLI](/ingest/pipeline/pipeline-cli)
and include the `--disable-hpa` flag:

```shell /PIPELINE/ theme={null}
calyptia update pipeline PIPELINE --disable-hpa
```

This flag resets the minimum and maximum replica counts, scaling policies, and
utilization targets for the pipeline, and deletes the autoscaler from the cluster.
The pipeline then runs with the [fixed number of replicas](#fixed-number-of-replicas)
set by its `--replicas` value.

You can't combine `--disable-hpa` with any other HPA flag in the same command. To
re-enable HPA later, run the `update pipeline` command again with the
`--min-replicas` and `--max-replicas` flags.

## Pause a pipeline

You can pause a pipeline by setting its replicas to zero, which temporarily
prevents that pipeline from routing or processing data. Use one of these methods
to pause a pipeline:

<Tabs>
  <Tab title="Web interface" id="pause-ui">
    1. Sign in to [Palo Alto Networks Telemetry Pipeline](https://core.calyptia.com/).
    2. Open the project that contains the pipeline that you want to pause.
    3. Go to **Core Instances**, then click the name of the Core Instance associated
       with the pipeline you want to pause.
    4. Under **Data Pipelines**, find the pipeline that you want to pause, and
       then click
       **<Icon icon="settings" /> Settings <span aria-label="and then">></span>
       <Icon icon="square-pause" /> Pause**.
    5. In the dialog that appears, click **Yes** to confirm your choice.
  </Tab>

  <Tab title="Pipeline CLI" id="pause-cli">
    In [Pipeline CLI](/ingest/pipeline/pipeline-cli), run the following command:

    ```shell /PIPELINE/ theme={null}
    calyptia update pipeline PIPELINE --replicas 0
    ```

    Replace *`PIPELINE`* with the name or [unique ID](/ingest/pipeline/v2/configure/metadata#get-pipeline-ids)
    of the pipeline you want to pause.
  </Tab>
</Tabs>

### Unpause a pipeline

If a pipeline is paused, use one of these methods to unpause it:

<Tabs>
  <Tab title="Web interface" id="unpause-ui">
    1. Sign in to [Palo Alto Networks Telemetry Pipeline](https://core.calyptia.com/).
    2. Open the project that contains the pipeline that you want to unpause.
    3. Go to **Core Instances**, then click the name of the Core Instance associated
       with the pipeline you want to unpause.
    4. Under **Data Pipelines**, find the pipeline that you want to unpause, and
       then click
       **<Icon icon="settings" /> Settings <span aria-label="and then">></span>
       <Icon icon="fast-forward" /> Resume**.
    5. In the dialog that appears, click **Yes** to confirm your choice.
  </Tab>

  <Tab title="Pipeline CLI" id="unpause-cli">
    In [Pipeline CLI](/ingest/pipeline/pipeline-cli), run the following command:

    ```shell /PIPELINE/ /REPLICAS/ theme={null}
    calyptia update pipeline PIPELINE --replicas REPLICAS
    ```

    Replace the following values:

    * *`PIPELINE`*: The name or [unique ID](/ingest/pipeline/v2/configure/metadata#get-pipeline-ids)
      of the pipeline you want to unpause.
    * *`REPLICAS`*: An integer value greater than zero.
  </Tab>
</Tabs>


## Related topics

- [Configure Palo Alto Networks Telemetry Pipeline](/ingest/pipeline/v2/configure.md)
- [Backpressure](/ingest/pipeline/v2/configure/backpressure.md)
- [Pipeline deployment strategies](/ingest/pipeline/v2/configure/deployment-strategies.md)
- [Azure metrics available in Palo Alto Networks Cortex XCOR](/ingest/xcor/integrations/server/azure/azure-metrics.md)
- [Kubernetes architecture of the CXDOT Collector](/ingest/xcor/collector/install/kubernetes/architecture.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.