> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Notifiers

> View and create notifiers with Palo Alto Networks Cortex XCOR, Chronoctl, or Terraform.

When an [alert](/investigate/alerts) triggers, Palo Alto Networks Cortex XCOR
sends a notification based on a *notifier*, which is an endpoint you specify that
defines where to deliver active alerts and who to notify. When creating a
[notification policy](/investigate/alerts/notifications/policies), you select a
notifier that specifies where to route the alert.

A single alert results in one notification, which can be of either `warn` or `critical`
severity. If there are multiple time series, the alert is de-duplicated and activated
only once. However, the alert indicates which time series are violating the condition.

Each notification policy triggers the sending of new notifications periodically,
with a default period of 60 minutes. Special [global notifiers](#global-notifiers)
receive and send notifications for every alert generated by Cortex XCOR.

For Slack, PagerDuty, webhooks, VictorOps, or OpsGenie, the
[Cortex XCOR API](/tooling/api-info) uses inline `destinations` that reference
[notifier connections](/investigate/alerts/notifications/external-connections),
or legacy `notifier_slugs` as in [Create a notifier](#create-a-notifier) and
[View notifiers](#view-notifiers). You can't set both on the same notifier list.
For rules, migration steps, and field-level destination fields, see
[Notifier resource migration](/investigate/alerts/notifications/external-connections#notifier-resource-migration)
and
[Notifier connections and route destinations](/investigate/alerts/notifications/policies#notifier-connections-and-route-destinations).

Choose a notifier based on where you want to deliver alerts. Cortex XCOR
supports the following notifiers:

<Columns cols={2} rowGroup="2">
  <Card horizontal icon="trash" title="Discard" href="/investigate/alerts/notifications/notifiers/discard">
    <p>Create a notifier to discard all notifications.</p>
  </Card>

  <Card horizontal icon="mail" title="Email" href="/investigate/alerts/notifications/notifiers/email">
    <p>Create an email notifier to send monitor alerts to a specific email address.</p>
  </Card>

  <Card horizontal icon="flame" title="incident.io" href="/investigate/alerts/notifications/notifiers/incident">
    <p>Configure an alert source in incident.io and then create a webhook notifier in Cortex XCOR.</p>
  </Card>

  <Card horizontal icon="atlassian" title="OpsGenie" href="/investigate/alerts/notifications/notifiers/opsgenie">
    <p>Create a notifier in OpsGenie.</p>
  </Card>

  <Card horizontal icon="circle-parking" title="PagerDuty" href="/investigate/alerts/notifications/notifiers/pagerduty">
    <p>Create a service in PagerDuty and then create the notifier in Cortex XCOR.</p>
  </Card>

  <Card horizontal icon="slack" title="Slack" href="/investigate/alerts/notifications/notifiers/slack">
    <p>Generate an incoming webhook in Slack and then create a Slack notifier in  Cortex XCOR.</p>
  </Card>

  <Card horizontal icon="leaf" title="VictorOps" href="/investigate/alerts/notifications/notifiers/victorops">
    <p>Create an API key in VictorOps and then create a VictorOps notifier in Cortex XCOR.</p>
  </Card>

  <Card horizontal icon="webhook" title="Webhook" href="/investigate/alerts/notifications/notifiers/webhook">
    <p>Create a webhook notifier with a URL that specifies the endpoint to send HTTP POST requests to.</p>
  </Card>
</Columns>

## Prerequisites

Some notifier configurations require specific information, such as a service key,
an API key, or support intervention. The required information depends on the type
of notifier that you're creating. Refer to each notifier's configuration to determine
the necessary information.

<Note>
  Users can modify Terraform-managed resources only by using Terraform.
  [Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
</Note>

## View notifiers

You can view available notifiers in Cortex XCOR, or return a list of
notifiers using Chronoctl. You can view notifiers created by Terraform in
Cortex XCOR, but you can't modify them.

<Tabs>
  <Tab title="Web" id="view-a-notifier-web">
    To view a notifier:

    1. In the navigation menu, select
       **<Icon icon="bell" /> Alerting <span aria-label="and then">></span> Notifiers**.
    2. Search for the notifier you want to view and select it from the list.

    The **Edit Notifier** page displays the notifier definition.

    If an entry has recent delivery failures, the list marks it with a **Delivery
    failing** badge. Hold the pointer over the badge to view the causes. Select the
    notifier to view banners with diagnostic messages, the number of affected
    monitors, and the failure duration. For remediation steps, see
    [Identify failed notification
    delivery](/investigate/alerts/notifications#identify-failed-notification-delivery).
  </Tab>

  <Tab title="Chronoctl" id="view-a-notifier-chronoctl">
    To view available notifiers with [Chronoctl](/tooling/chronoctl), use one of the following commands:

    * To return a complete list of notifier resource definitions as YAML documents, use
      this command:

      ```shell theme={null}
      chronoctl notifiers list
      ```

    * To return specific resources by name or slug, pass a
      comma-delimited list to the `--names` or `--slugs` options respectively. For
      example:

      ```shell /LIST/ theme={null}
      chronoctl notifiers list --names LIST
      ```

      Replace *`LIST`* with a list of notifier names, such as `"Infra Webhook,Email Notifier"`.
      Each notifier has a `name` field in the resource definition, which you include in
      the list.

    * To return a single resource definition by slug, use the `notifiers read` command:

      ```shell /SLUG/ theme={null}
      chronoctl notifiers read SLUG
      ```

      Replace *`SLUG`* with the notifier's slug, such as `email-notifier`.

    These commands return a `Notifier` resource, which has the following format:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      slug: email-notifier
      name: Email Notifier
      created_at: "2022-11-17T22:56:59.000Z"
      updated_at: "2022-11-17T22:56:59.000Z"
      email:
        to: your-address@example.com
        html: <b>Alert triggering</b>
        text: Alert triggering
    ```
  </Tab>

  <Tab title="API" id="view-a-notifier-API">
    To complete this action with the Cortex XCOR API, use the
    [`ListNotifiers`](/tooling/api-info/definition/operations/ListNotifiers) endpoint.

    Because the Cortex XCOR API requires authentication, include an API token with your
    `curl` request, as shown in the following example. For more details, see
    [Create an API token](/tooling/api-info#create-an-api-token).

    ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
    export CHRONOSPHERE_API_TOKEN="TOKEN"
    export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

    curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
         -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
    ```

    Replace the following:

    * *`TOKEN`*: Your API token.
    * *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
    * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
    * *`ENDPOINT_PATH`*: The specific endpoint you want to access.
  </Tab>
</Tabs>

## Create a notifier

Notifiers use a custom definition format in YAML. You can use Cortex XCOR,
Chronoctl, or the Terraform provider to create and manage them.
Notifiers support [variables](#use-variables-in-notifiers) and templating.

After creating a notifier, you can select it when defining a
[notification policy](/investigate/alerts/notifications/policies).

The following steps apply to creating notifiers. See each notifier page for examples
of creating that specific notifier type.

<Tabs>
  <Tab title="Web" id="create-a-notifier-web">
    To create a notifier:

    1. In the navigation menu, select
       **<Icon icon="bell" /> Alerting <span aria-label="and then">></span> Notifiers**.
    2. Click **Create notifier**.
    3. Enter a descriptive name for the notifier.
    4. Select the type of notifier you want to create.
    5. Enter the required information for the notifier.
    6. Optional: Select **Notify when resolved** to send a resolved alert notification.
    7. Click **Save**.
  </Tab>

  <Tab title="Chronoctl" id="create-a-notifier-chronoctl">
    To create one or more notifiers with [Chronoctl](/tooling/chronoctl), create a YAML
    file to define your notifiers and apply the configuration to your instance. Use the
    lowercase name of the service as a YAML key to set the service to use.

    Refer to each notifier type for a full example of a definition. You can also generate
    an example that contains multiple notifier types with Chronoctl by using this
    command:

    ```shell theme={null}
    chronoctl notifiers scaffold
    ```

    To apply a definition, use this command:

    ```shell /FILE_NAME/ theme={null}
    chronoctl apply -f FILE_NAME.yaml
    ```

    Replace *`FILE_NAME`* with the name of your notifier YAML file.
  </Tab>

  <Tab title="Terraform" id="create-a-notifier-terraform">
    Create an alert notifier with Terraform by using the
    `chronosphere_{channel}_alert_notifier` type followed by a name in a resource
    declaration.

    See each notifier type for a full definition example.

    1. Add the definition to a Terraform file.
    2. Run this command to create the resource:

       ```shell theme={null}
       terraform apply
       ```
  </Tab>

  <Tab title="API" id="create-a-notifier-API">
    To complete this action with the Cortex XCOR API, use the
    [`CreateNotifier`](/tooling/api-info/definition/operations/CreateNotifier) endpoint.

    Because the Cortex XCOR API requires authentication, include an API token with your
    `curl` request, as shown in the following example. For more details, see
    [Create an API token](/tooling/api-info#create-an-api-token).

    ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
    export CHRONOSPHERE_API_TOKEN="TOKEN"
    export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

    curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
         -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
    ```

    Replace the following:

    * *`TOKEN`*: Your API token.
    * *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
    * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
    * *`ENDPOINT_PATH`*: The specific endpoint you want to access.
  </Tab>
</Tabs>

<Note>
  To create [global notifiers](#global-notifiers), contact [Cortex XCOR Support](/support).
</Note>

## Edit a notifier

Select from the following methods to edit a notifier.

<Tabs>
  <Tab title="Web" id="edit-a-notifier-web">
    <Note>
      Users can modify Terraform-managed resources only by using Terraform.
      [Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
    </Note>

    To edit a notifier:

    1. In the navigation menu, select
       **<Icon icon="bell" /> Alerting <span aria-label="and then">></span> Notifiers**.
    2. Search for the notifier you want to view and select it from the list.
    3. In the **Edit Notifier** page, make changes to the notifier definition.
    4. Click **Save**.
  </Tab>

  <Tab title="Chronoctl" id="edit-a-notifier-chronoctl">
    <Note>
      Users can modify Terraform-managed resources only by using Terraform.
      [Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
    </Note>

    To edit a notifier with [Chronoctl](/tooling/chronoctl):

    1. Get the resource definition for the notifier you want to edit:

       * To return a complete list of notifier resource definitions as YAML documents, use
         the `notifiers list` command:

         ```shell theme={null}
         chronoctl notifiers list
         ```

       * To return a single resource definition by slug, use this command:

         ```shell /SLUG/ theme={null}
         chronoctl notifiers read SLUG
         ```

         Replace *`SLUG`* with the notifier's slug, such as `email-notifier`.

    2. Modify the YAML file for the notifier you want to update.

    3. Apply your changes:

       ```shell /FILE_NAME/ theme={null}
       chronoctl apply -f FILE_NAME.yaml
       ```

       Replace *`FILE_NAME`* with the name of your notifier YAML file.

    Chronoctl updates the notifier's properties if it has the same slug.
  </Tab>

  <Tab title="Terraform" id="edit-a-notifier-terraform">
    To edit a notifier using [Terraform](/tooling/infrastructure/terraform):

    1. Modify the notifier resource's existing properties.
    2. Run `terraform apply` to apply the changes.

       ```shell theme={null}
       terraform apply
       ```
  </Tab>

  <Tab title="API" id="edit-a-notifier-API">
    To complete this action with the Cortex XCOR API, use the
    [`UpdateNotifier`](/tooling/api-info/definition/operations/UpdateNotifier) endpoint.

    Because the Cortex XCOR API requires authentication, include an API token with your
    `curl` request, as shown in the following example. For more details, see
    [Create an API token](/tooling/api-info#create-an-api-token).

    ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
    export CHRONOSPHERE_API_TOKEN="TOKEN"
    export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

    curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
         -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
    ```

    Replace the following:

    * *`TOKEN`*: Your API token.
    * *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
    * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
    * *`ENDPOINT_PATH`*: The specific endpoint you want to access.
  </Tab>
</Tabs>

<Note>
  To edit [global notifiers](#global-notifiers), contact [Cortex XCOR Support](/support).
</Note>

## Delete a notifier

Select from the following methods to delete a notifier.

<Tabs>
  <Tab title="Web" id="delete-a-notifier-web">
    <Note>
      Users can modify Terraform-managed resources only by using Terraform.
      [Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
    </Note>

    To delete a notifier:

    1. In the navigation menu, select
       **<Icon icon="bell" /> Alerting <span aria-label="and then">></span> Notifiers**.
    2. Search for the notifier you want to delete and select it from the list.
    3. In the **Edit Notifier** page, click the delete icon **<Icon icon="trash" />**.
    4. In the confirmation dialog, click **Delete**.
  </Tab>

  <Tab title="Chronoctl" id="delete-a-notifier-chronoctl">
    <Note>
      Users can modify Terraform-managed resources only by using Terraform.
      [Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
    </Note>

    To delete a notifier with [Chronoctl](/tooling/chronoctl), run this command:

    ```shell /SLUG/ theme={null}
    chronoctl notifiers delete SLUG
    ```

    Replace *`SLUG`* with the slug of the notifier you want to delete.
  </Tab>

  <Tab title="Terraform" id="delete-a-notifier-terraform">
    To delete a resource that's managed by [Terraform](/tooling/infrastructure/terraform):

    1. Edit your Terraform configuration file to remove the pre-existing resource
       definition.
    2. Run this command to remove the resource from Cortex XCOR:

       ```shell theme={null}
       terraform apply
       ```
  </Tab>

  <Tab title="API" id="delete-a-notifier-API">
    To complete this action with the Cortex XCOR API, use the
    [`DeleteNotifier`](/tooling/api-info/definition/operations/DeleteNotifier) endpoint.

    Because the Cortex XCOR API requires authentication, include an API token with your
    `curl` request, as shown in the following example. For more details, see
    [Create an API token](/tooling/api-info#create-an-api-token).

    ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
    export CHRONOSPHERE_API_TOKEN="TOKEN"
    export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

    curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
         -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
    ```

    Replace the following:

    * *`TOKEN`*: Your API token.
    * *`INSTANCE`*: The subdomain name for your organization's Cortex XCOR instance.
    * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
    * *`ENDPOINT_PATH`*: The specific endpoint you want to access.
  </Tab>
</Tabs>

<Note>
  To delete [global notifiers](#global-notifiers), contact [Cortex XCOR Support](/support).
</Note>

## Notifications for alerts in third-party services

By default, Cortex XCOR sends a notification when an alert for a monitor
resolves in third-party services, such as PagerDuty and VictorOps. You can choose to
not send notifications by configuring the specified parameter in the Chronoctl YAML
definition or Terraform resource for your notifier.

<Note>
  By default, Cortex XCOR doesn't send notifications when an alert resolves.
  To send resolved notifications, select **Notify when resolved** when creating a notifier.
  For notifiers created using Chronoctl, Terraform, or the Cortex XCOR API, use the
  `skip_resolved` or `send_resolved` parameter.
</Note>

<Tabs>
  <Tab title="Chronoctl" id="resolve-alerts-in-third-party-services-chronoctl">
    To skip notifications when alerts for a monitor resolve, set the value of the
    `skip_resolved` property to `true` in your notifier definition.

    For example, the following definition skips sending notifications for resolved alerts
    for an Opsgenie notifier:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: test-opsgenie
      slug: test-opsgenie
      skip_resolved: true
      ops_genie:
        # ...
    ```
  </Tab>

  <Tab title="Terraform" id="resolve-alerts-in-third-party-services-terraform">
    To skip notifications when alerts for a monitor resolve, set the value of the
    `send_resolved` key to `false`. If you don't explicitly set a value for this key,
    Cortex XCOR sends notifications for resolved alerts by default. The
    `send_resolved` key is a top-level key.

    For example, the following resource skips sending notifications for resolved alerts
    for an Opsgenie notifier:

    ```terraform Terraform example icon="square-terminal" theme={null}
    resource "chronosphere_opsgenie_alert_notifier" "opsgenie_notifier" {
      name = "Opsgenie Notifier"
      # ...
      send_resolved = false
      # ...
    }
    ```
  </Tab>
</Tabs>

## Use variables in notifiers

Notification content supports [Go templates](https://pkg.go.dev/text/template@go1.19) and
[Prometheus Alertmanager variables](https://prometheus.io/docs/alerting/latest/notifications/).
Notifiers support many features and variables demonstrated in
[Alertmanager notification examples](https://prometheus.io/docs/alerting/latest/notification_examples/).

### Available variables

You can reference Prometheus Alertmanager variables with the `{{.VARIABLE_NAME }}`
syntax. Variable references work in the notifier's definition and the notification's
contents.

Notifiers can access monitor labels by using variables with the
`{{ .CommonLabels.LABEL }}` pattern, and from the alerting metric with the
`{{ .Labels.LABEL }}` pattern. In both patterns, replace *`LABEL`* with the
label's name.

Similarly, you can reference monitor-defined annotations by using the
`{{ .CommonAnnotations.ANNOTATION }}` pattern, replacing *`ANNOTATION`* with
the monitor annotation's name.

<Note>
  If your monitor query includes a label that generates different annotation values,
  then annotations won't be included in `CommonAnnotations` or `CommonLabels`.
</Note>

See the
[Alertmanager documentation](https://prometheus.io/docs/alerting/latest/notifications/)
for a reference list of alerting variables and templating functions.

For a simpler way to customize the title and description of notifications
using monitor-specific variables, see
[Notification templates](/investigate/alerts/monitors/notification-templates).

### Variable usage examples

This Slack notifier's `channel` value takes advantage of monitor-defined labels to
post a notification to a channel based on a triggering metric's label:

<Tabs>
  <Tab title="Chronoctl" id="variable-usage-examples-1-chronoctl">
    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: slack-notifier
      slug: slack-notifier
      slack:
        # ...
        channel: "metric-notification-{{ .CommonLabels.<LABEL> }}"
        # ...
    ```
  </Tab>

  <Tab title="Terraform" id="variable-usage-examples-1-terraform">
    ```terraform Terraform example icon="square-terminal" theme={null}
    resource "chronosphere_slack_alert_notifier" "slack_notifier" {
      name = "Slack Notifier"
      # ...
      channel = "metric-notification-{{ .CommonLabels.<LABEL> }}"
      # ...
    }
    ```
  </Tab>
</Tabs>

You can also reference annotations defined in the monitor:

<Tabs>
  <Tab title="Chronoctl" id="variable-usage-examples-2-chronoctl">
    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: slack-notifier
      slug: slack-notifier
      slack:
        # ...
        text: "Summary: {{ .CommonAnnotations.summary }}"
        # ...
    ```
  </Tab>

  <Tab title="Terraform" id="variable-usage-examples-2-terraform">
    ```terraform Terraform example icon="square-terminal" theme={null}
    resource "chronosphere_slack_alert_notifier" "slack_notifier" {
      name = "Slack Notifier"
      # ...
      text = "Summary: {{ .CommonAnnotations.summary }}"
      # ...
    }
    ```
  </Tab>
</Tabs>

You can also use multiple labels in notifiers. The following example uses multiple
labels for an OpsGenie notifier:

<Tabs>
  <Tab title="Chronoctl" id="variable-usage-examples-3-chronoctl">
    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      ops_genie:
        priority: '{{ .CommonLabels.Priority }}'
    ---
    api_version: v1/config
    kind: Monitor
    spec:
      labels:
        Priority: P2
    ```

    The priority in the outgoing notification is correctly templated with priority `P2`.
  </Tab>
</Tabs>

## Global notifiers

Instead of using a notification policy to define when a notification is sent, global notifiers
receive and send notifications for every alert generated across Cortex XCOR.

Since these notifiers can generate a constant feed of many notifications, they must
be created and managed exclusively by [Cortex XCOR Support](/support). You can't
edit or delete global notifiers on your own, and Cortex XCOR designates
this with a banner in notifier editing interfaces.

In the Notifiers list, Cortex XCOR designates global notifiers with a
**Global** chip displayed with their type.


## Related topics

- [Notifier connections](/investigate/alerts/notifications/external-connections.md)
- [Create a discard notifier](/investigate/alerts/notifications/notifiers/discard.md)
- [UpdateNotifier](/tooling/api-info/definition/operations/UpdateNotifier.md)
- [CreateNotifier](/tooling/api-info/definition/operations/CreateNotifier.md)
- [ReadNotifier](/tooling/api-info/definition/operations/ReadNotifier.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.