Configure Palo Alto Networks Telemetry Pipeline using Helm

This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone product separate from Palo Alto Networks Cortex XCOR.

If you used Helm to install Core Operators and create Core Instances, you can also use Helm to configure your Palo Alto Networks Telemetry Pipeline installation and to perform various pipeline operations.

Specify configurations

You can use the helm upgrade -i shell command to specify your configuration by passing a YAML file or specific flags to the command.

YAML file
Shell command

Provide a YAML file that specifies the values of parameters while installing the chart. For example:

helm upgrade -i core-instance calyptia/core-instance --values FILE

Replace FILE with the name of your YAML file.

Specify each parameter using helm upgrade -i, and supply arguments using --set key=value[,key=value]. For example:

helm upgrade -i core-instance calyptia/core-instance --set name="core-instance" --set cloudToken="CLOUD_TOKEN" --set coreInstance="CORE_INSTANCE"

Replace CLOUD_TOKEN and CORE_INSTANCE in accordance with the parameter details.

Container image configuration

Palo Alto Networks Telemetry Pipeline uses Bitnami conventions to configure container images.

Global settings

To set global settings for your container images, use the global.imageRegistry parameter:

global:
  imageRegistry: ""
  imagePullSecrets: []
  storageClass: ""
  pullPolicy: IfNotPresent

If the global container registry requires authentication, you can provide pull secrets in the global.imagePullSecrets parameter.

Individual settings

To set individual settings for your container images, find that image’s key nested under the images key. For example, you can modify the fromCloud image using the parameters in images.fromCloud:

images:
  fromCloud:
    registry: ghcr.io
    repository: calyptia/core-operator/sync-from-cloud
    tag: 3.23.0
    pullSecrets: []

If a container image requires authentication, you can provide pull secrets in that image’s pullSecrets parameter. that image’s pullSecrets parameter. For example, to include pull secrets in the fromCloud container, store their values in images.fromCloud.pullSecrets.

Services and ports

The Helm chart for Palo Alto Networks Telemetry Pipeline provides the following Kubernetes Services:

cloud-api: Corresponds to the Palo Alto Networks Telemetry Pipeline backend (cloud-api.calyptia.com). Default port: 5000.
core: Corresponds to the Palo Alto Networks Telemetry Pipeline web interface (core.calyptia.com). Default port: 3000.

Both of these Kubernetes Services use the LoadBalancer Service type by default.

Troubleshooting

If you need assistance, contact Cortex XCOR Support. As part of the support process, Cortex XCOR Support recommends downloading and running the support.sh script, which gathers information about your Palo Alto Networks Telemetry Pipeline installation and packages this information into an archive that you can attach to your ticket. However, be aware that the output generated by the support.sh script can include any information related to your Palo Alto Networks Telemetry Pipeline installation, including the values of secrets or authentication credentials you’ve stored. Before sending this output to Cortex XCOR Support, be sure to redact any sensitive information accordingly.

Parameter details

The following sections list the configuration parameters of the Palo Alto Networks Telemetry Pipeline Helm charts and their default values. A default value of "" indicates an empty string. Palo Alto Networks Telemetry Pipeline uses three Helm charts in combination:

core-crd
core-operator
core-instance

`core-crd` parameters

The core-crd chart packages the Custom Resource Definitions (CRDs) used by the Palo Alto Networks Telemetry Pipeline operator, also known as a Core Operator.

Common

These parameters configure general settings:

Parameter	Description	Default value
`enabled`	If `true`, enables this chart. If `false`, disables this chart.	`true`
`pipelineServiceType`	Specifies the default Service type to use for new pipelines.	`LoadBalancer`

Images

The parameters in this table configure the default images for pipelines and ingest checks:

Parameter	Description	Default value
`images.hotReload.registry`	The registry that stores the container image to use for pipelines during a hot reload.	`ghcr.io`
`images.hotReload.repository`	The repository that stores the container image to use for pipelines during a hot reload.	`calyptia/configmap-reload`
`images.hotReload.tag`	The tag associated with the container image to use for pipelines during a hot reload. To use the Red Hat UBI agent image, provide only a version number (for example, `1.2.3`). To use the distroless agent image, append `-distroless` to the version number (for example, `1.2.3-distroless`).	latest version
`images.fluentBit.registry`	The registry that stores the standard container image to use for pipelines.	`ghcr.io`
`images.fluentBit.repository`	The repository that stores the standard container image to use for pipelines.	`calyptia/core/calyptia-fluent-bit`
`images.fluentBit.tag`	The tag associated with the standard container image to use for pipelines. To use the Red Hat UBI agent image, provide only a version number (for example, `1.2.3`). To use the distroless agent image, append `-distroless` to the version number (for example, `1.2.3-distroless`).	latest version

`core-operator` parameters

The core-operator chart packages the Palo Alto Networks Telemetry Pipeline operator, also known as a Core Operator.

Global image

The parameters in this table configure the settings for pipeline images:

Parameter	Description	Default value
`global.imageRegistry`	The registry that stores pipeline images. This value overrides any registries set within associated Core Instances.	`""`
`global.imagePullSecrets`	An array of the key names for pull secrets to use across all images. Any values in this array are added to the list of pull secret names specified in each image’s `pullSecrets` parameter; they don’t override per-image settings.	`["regcreds"]`
`global.pullPolicy`	Sets the pull policy used by Kubernetes. If a value is specified, this setting overrides the associated `pullPolicy` in any per-image settings.	`""`

Deployment

The parameters in this table configure the Core Operator:

Parameter	Description	Default value
`enabled`	If `true`, enables this chart. If `false`, disables this chart.	`true`
`images.operator.registry`	The registry that stores the operator image. This value can be overridden by `global.imageRegistry`.	`ghcr.io`
`images.operator.repository`	The repository that stores the operator image.	`calyptia/core-operator`
`images.operator.tag`	The tag associated with the operator image.	latest version
`images.operator.pullSecrets`	An array of the key names for pull secrets to use in the specified image. Any values in this array are added to the list of pull secret names specified in `global.image.pullSecrets`.	`[]`
`images.operator.pullPolicy`	Sets the pull policy used by Kubernetes for the specified image. This value can be overwritten by a value set in `global.image.pullPolicy`.	`IfNotPresent`
`commonAnnotations`	Specifies annotations to add to all resources, with the exception of the Core Operator Pod.	`{}`
`podAnnotations`	Specifies annotations to add to the Core Operator Pod.	`{}`
`namespaceOverride`	If a value is set, this parameter overrides the namespace specified by the `.Release.Namespace` variable.	`""`
`calyptiaTolerations`	Specifies tolerations to add to the Core Operator Pod.	`""`
`restartPolicy`	Sets the restart policy for the Core Operator Pod.	`Always`

Resources

The parameters in this table define resources for the Core Operator:

Parameter	Description	Default value
`resources.limits.cpu`	Sets the Core Operator’s CPU limit.	`500m`
`resources.limits.memory`	Sets the Core Operator’s memory limit.	`512Mi`
`resources.requests.cpu`	Sets the Core Operator’s CPU request.	`10m`
`resources.requests.memory`	Sets the Core Operator’s memory limit.	`64Mi`

Probes

The parameters in this table configure probes to monitor the health of the Core Operator:

Parameter	Description	Default value
`readinessProbe.httpGet.path`	The HTTP path of the readiness probe.	`/readyz`
`readinessProbe.httpGet.port`	The port associated with the readiness probe.	`8081`
`readinessProbe.initialDelaySeconds`	The initial delay for the readiness probe, in seconds.	`5`
`readinessProbe.periodSeconds`	Specifies how often to run the readiness probe, in seconds.	`10`
`livenessProbe.httpGet.path`	The HTTP path of the liveness probe.	`/healthz`
`livenessProbe.httpGet.port`	The port associated with the liveness probe.	`8081`
`livenessProbe.initialDelaySeconds`	Specifies the initial delay for the liveness probe, in seconds.	`15`
`livenessProbe.periodSeconds`	Specifies how often to run the liveness probe, in seconds.	`20`

RBAC

The parameters in this table configure role-based access control (RBAC) settings:

Parameter	Description	Default value
`rbac.create`	Specifies whether to create RBAC resources.	`true`
`serviceAccount.create`	If `true`, create a ServiceAccount for the operator. If `false`, don’t create a ServiceAccount.	`true`
`serviceAccount.name`	If `serviceAccount.create` is `true`, specifies the name to give that ServiceAccount.	`""`
`serviceAccount.annotations`	Specifies annotations to add to the ServiceAccount.	`{}`
`fullnameOverride`	If set, overrides the default value of the `serviceAccount.name` parameter.	`""`
`nameOverride`	If set, overrides the default value of the `fullnameOverride` parameter.	`""`

Pull secrets

The parameters in this table configure the secrets used to pull images from a private registry:

Parameter	Description	Default value
`imageCredentials.secretName`	The name of the secret. This value must refer to an existing `pullSecret`.	none
`imageCredentials.registry`	The image registry. This value must match the `registry` value of the images you’re trying to pull.	none
`imageCredentials.username`	The username to sign in to the registry.	none
`imageCredentials.password`	The password to sign in to the registry.	none
`imageCredentials.email`	The email address to sign in to the registry.	none

`core-instance` parameters

The core-instance chart governs an associated Core Instance, which syncs Palo Alto Networks Telemetry Pipeline between your local environment and the Palo Alto Networks Telemetry Pipeline backend.

Global image

The parameters in this table configure the settings for pipeline images:

Parameter	Description	Default value
`global.imageRegistry`	The registry that stores pipeline images. This value overrides any registries set within associated Core Instances.	`""`
`global.imagePullSecrets`	An array of the key names for pull secrets to use across all images. Any values in this array are added to the list of pull secret names specified in each image’s `pullSecrets` parameter; they don’t override per-image settings.	`["regcreds"]`
`global.pullPolicy`	Sets the pull policy used by Kubernetes. If a value is specified, this setting overrides the associated `pullPolicy` in any per-image settings.	`""`

Deployment

The parameters in this table configure the Core Instance:

Parameter	Description	Default value
`enabled`	If `true`, enables this chart. If `false`, disables this chart.	`true`
`coreInstance`	Sets the name of the Core Instance.	`""`
`cloudToken`	Your Palo Alto Networks Telemetry Pipeline API token in plain text. Only use this parameter if `cloudTokenSec.name` and `cloudTokenSec.key` aren’t set.	`""`
`cloudTokenSec.name`	The name of the Kubernetes secret that stores your Palo Alto Networks Telemetry Pipeline API token. Only use this parameter if `cloudToken` isn’t set.	`nil`
`cloudTokenSec.key`	The key of the Kubernetes secret that stores your Palo Alto Networks Telemetry Pipeline API token. Only use this parameter if `cloudToken` isn’t set.	`nil`
`cloudUrl`	The URL of the Palo Alto Networks Telemetry Pipeline backend.	`https://cloud-api.calyptia.com`
`images.fromCloud.registry`	The registry that stores the `fromCloud` container image, which is a synchronization container that communicates with the Palo Alto Networks Telemetry Pipeline backend. This value can be overridden by `global.imageRegistry`.	`ghcr.io`
`images.fromCloud.repository`	The repository that stores the `fromCloud` container image.	`calyptia/core-operator/sync-from-cloud`
`images.fromCloud.tag`	The tag associated with the `fromCloud` container image.	latest version
`images.fromCloud.pullSecrets`	The names of the pull secrets to use to fetch the `fromCloud` image. Any values in this array are added to the list of pull secret names specified in `global.image.pullSecrets`.	`[]`
`images.fromCloud.pullPolicy`	Sets the pull policy for the `fromCloud` container image. This value can be overridden by `global.image.pullPolicy`.	`IfNotPresent`
`images.toCloud.registry`	The registry that stores the `toCloud` container image, which is a synchronization container that communicates with the Palo Alto Networks Telemetry Pipeline backend. This value can be overridden by `global.imageRegistry`.	`ghcr.io`
`images.toCloud.repository`	The repository that stores the `fromCloud` container image.	`calyptia/core-operator/sync-to-cloud`
`images.toCloud.tag`	The tag associated with the `toCloud` container image.	latest version
`images.toCloud.pullSecrets`	The names of the pull secrets to use to fetch the `toCloud` image. Any values in this array are added to the list of pull secret names specified in `global.image.pullSecrets`.	`[]`
`images.toCloud.pullPolicy`	Sets the pull policy for the `toCloud` container image. This value can be overridden by `global.image.pullPolicy`.	`IfNotPresent`
`images.hotReload.registry`	The registry that stores the container image to use for pipelines during a hot reload. This value can be overridden by `global.imageRegistry`.	`ghcr.io`
`images.hotReload.repository`	The repository that stores the container image to use for pipelines during a hot reload.	`calyptia/configmap-reload`
`images.hotReload.tag`	The tag associated with the container image to use for pipelines during a hot reload.	latest version
`interval`	Specifies how often to sync data to and from the Palo Alto Networks Telemetry Pipeline backend. Possible units of measurement are `ns` (nanoseconds), `us` or `µs` (microseconds), `ms` (milliseconds), `s` (seconds), `m` (minutes), and `h` (hours).	`15s`
`probeIntervalSec`	Specifies how often to run liveness and readiness probes, in seconds. If set to `0`, both probes are disabled.	`5`
`commonAnnotations`	Specifies annotations to add to the Core Instance Pods created by this chart, but not to pipeline Pods. To add annotations to pipeline Pods, use `calyptiaAnnotations`. This value must follow `annotation1=value1,annotation2=value2` syntax.	`{}`
`namespaceOverride`	If a value is set, this parameter overrides the namespace specified by the `.Release.Namespace` variable.	`""`
`fromCloud.debugPort`	The debug port for the `fromCloud` container.	`5334`
`fromCloud.resources`	The resources for the `fromCloud` container.	`{}`
`toCloud.debugPort`	The debug port for the `toCloud` container.	`15334`
`toCloud.resources`	The resources for the `toCloud` container.	`{}`
`cloudTimeout`	Specifies the HTTP timeout for requests to the Palo Alto Networks Telemetry Pipeline backend, in seconds. If set to `0`, the timeout is disabled.	`0`

Pipeline

The parameters in this table configure pipelines deployed within the associated Core Instance:

Parameter	Description	Default value
`clusterLogging`	If `true`, enables automated Kubernetes logging. If `false`, disables automated Kubernetes logging.	`false`
`calyptiaAnnotations`	Specifies annotations to add to pipeline Pods. This value must follow `annotation1=value1,annotation2=value2` syntax. However, if you’re using Helm through the command line, you must escape any commas with a backslash (`\`) to prevent Helm from incorrectly splitting the string into multiple values. For example, `--set calyptiaAnnotations='annotation1=value1\,annotation2=value2'` is valid syntax.	`""`
`calyptiaTolerations`	Specifies tolerations to add to pipeline Pods. Only use this setting if you haven’t already configured per-pipeline tolerations. This value must follow the toleration string syntax.	`""`
`enableHealthCheckPipeline`	If `true`, creates a health check pipeline in the Core Instance. If `false`, doesn’t create a health check pipeline.	`false`
`healthCheckPipelinePort`	The frontend port for the health check pipeline.	`2020`
`healthCheckPipelineServiceType`	The Service type to use for the health check pipeline.	`LoadBalancer`
`skipServiceCreation`	If `true`, no Service is created for the pipeline. If `false`, creates a Service for the pipeline.	`false`
`notls`	If `true`, disables TLS verification for pipeline configuration settings. If `false`, enables TLS verification for pipeline configuration settings.	`true`
`httpProxy`	Sets the `HTTP_PROXY` environment variable for pipeline containers.	`""`
`httpsProxy`	Sets the `HTTPS_PROXY` environment variable for pipeline containers.	`""`
`noProxy`	Sets the `NO_PROXY` environment variable for pipeline containers.	`""`
`cloudProxy`	Specifies the HTTP proxy to use when connecting to the Palo Alto Networks Telemetry Pipeline backend.	`""`

RBAC

The parameters in this table configure role-based access control (RBAC) settings:

Parameter	Description	Default value
`rbac.create`	If `true`, creates RBAC resources. If `false`, doesn’t create RBAC resources.	`true`
`serviceAccount.create`	If `true`, creates a ServiceAccount for the operator. If `false`, doesn’t create a ServiceAccount.	`true`
`serviceAccount.name`	If `serviceAccount.create` is `true`, specifies the name to give that ServiceAccount.	`""`
`serviceAccount.annotations`	Specifies annotations to add to the ServiceAccount.	`{}`
`fullnameOverride`	If set, overrides the default value of the `serviceAccount.name` parameter.	`""`
`nameOverride`	If set, overrides the default value of the `fullnameOverride` parameter.	`""`

Pull secrets

The parameters in this table configure the secrets used to pull images from a private registry:

Parameter	Description	Default value
`imageCredentials.secretName`	The name of the secret. This value must refer to an existing `pullSecret`.	none
`imageCredentials.registry`	The image registry. This value must match the `registry` value of the images you’re trying to pull.	none
`imageCredentials.username`	The username to log in to the registry.	none
`imageCredentials.password`	The password to log in to the registry.	none
`imageCredentials.email`	The email address to log in to the registry.	none

​Specify configurations

​Container image configuration

​Global settings

​Individual settings

​Services and ports

​Troubleshooting

​Parameter details

​core-crd parameters

​Common

​Images

​core-operator parameters

​Global image

​Deployment

​Resources

​Probes

​RBAC

​Pull secrets

​core-instance parameters

​Global image

​Deployment

​Pipeline

​RBAC

​Pull secrets

Specify configurations

Container image configuration

Global settings

Individual settings

Services and ports

Troubleshooting

Parameter details

`core-crd` parameters

Common

Images

`core-operator` parameters

Global image

Deployment

Resources

Probes

RBAC

Pull secrets

`core-instance` parameters

Global image

Deployment

Pipeline

RBAC

Pull secrets