> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Elasticsearch

> Collect cluster, node, index, operating system, process, and JVM metrics from Elasticsearch.

The Elasticsearch integration requires CXDOT Collector 1.7.0 or greater.

[Elasticsearch](https://www.elastic.co/elasticsearch) is a distributed search and
analytics engine. Use the Elasticsearch integration to collect cluster, node,
operating system, process, and Java Virtual Machine metrics from Elasticsearch
servers running in your environment.

The Elasticsearch integration supports Elasticsearch 7.9 or later.

## Supported telemetry types

The Elasticsearch integration supports these telemetry types:

| Type | Supported |
| - | - |
| Logs | No |
| Metrics | Yes |
| Traces | No |
| Events | No |

## Prerequisites

The Elasticsearch integration has the following prerequisites:

* If Elasticsearch security is enabled, create a monitoring user with the `monitor`
  or `manage` cluster privilege. The `monitor` privilege provides the required
  read-only access. For more information, see
  [Elasticsearch security privileges](https://www.elastic.co/docs/deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges).
* Make the Elasticsearch HTTP API reachable from the CXDOT Collector.

## Configure

The Elasticsearch integration is enabled by default. To configure the integration,
follow these steps:

1. For Elasticsearch pods that expose an unsecured HTTP API on port `9200`, add the
   `app.kubernetes.io/name: elasticsearch` label to the pod template. The CXDOT
   Collector also discovers pods that carry autodiscovery annotations for
   Elasticsearch. Use annotations to supply credentials, an HTTPS URL, or a custom
   port for each target.

   Metrics from discovered pods arrive with that pod's Kubernetes metadata attached.
   For more information, see
   [autodiscovery](https://docs.paloaltonetworks.com/cortex/xcor/ingest/xcor/collector/discover/kubernetes).

2. Optional: Configure static targets instead of discovered targets, such as
   Elasticsearch servers outside your Kubernetes cluster. Supply the password
   through your deployment's secret management. For example, add the following to
   the `values.yaml` for your CXDOT Collector Helm chart:

   ```yaml theme={null}
   config:
     integrations:
       elasticsearch:
         endpoints:
           - endpoint: https://elasticsearch.example.com:9200
         username: cxdot
         password: "${env:ELASTICSEARCH_PASSWORD}"
   ```

   When you configure static targets, this integration instance collects from
   exactly those targets instead of discovered targets.

3. Optional: Configure a second integration instance to collect from both discovered
   pods and static targets. For example, add the following to the `values.yaml` for
   your CXDOT Collector Helm chart:

   ```yaml theme={null}
   config:
     integrations:
       elasticsearch: {}
       elasticsearch/edge:
         endpoints:
           - endpoint: elasticsearch.example.com:9200
   ```

   The `elasticsearch/edge` key defines a named instance for the static target. The
   bare `elasticsearch` key continues to use autodiscovery.

4. Optional: Disable the integration. For example, add the following to the
   `values.yaml` for your CXDOT Collector Helm chart:

   ```yaml theme={null}
   config:
     integrations:
       elasticsearch:
         enabled: false
   ```

### Validate

To validate the Elasticsearch integration, follow these steps:

1. In the [Live Telemetry Analyzer](https://docs.paloaltonetworks.com/cortex/xcor/investigate/analyze/telemetry-analyzer),
   filter for `__name__=cxdot.integration.target.health` and
   `cxdot.integration.name=elasticsearch`. Confirm that each reachable Elasticsearch
   target reports `1`, identified by its `server.address` and `server.port`
   attributes.

2. In [Metrics Explorer](https://docs.paloaltonetworks.com/cortex/xcor/investigate/querying/metrics/explorer),
   run the following query after Elasticsearch has indexed documents:

   ```text theme={null}
   sum by ("server.address", "server.port") ({"elasticsearch.node.documents", "state"="active"})
   ```

   Confirm that the query returns the expected document count for each target.

## Configuration reference

Configure one Elasticsearch integration instance with the following settings. In Helm values,
place these settings under `config.integrations.elasticsearch`. In a Collector configuration
file, place them under `cxdot.integrations.elasticsearch`.

### Optional settings

* **`enabled`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to enable this Elasticsearch integration instance. If true, the Collector collects
  Elasticsearch metrics. If false, the Collector doesn't run this integration instance.

* **`endpoints`**
  Type: `array of object`. Optional. Default: `[]`.
  Static Elasticsearch targets. Each `endpoint` is either `host:port` for an HTTP connection or
  a full HTTP or HTTPS URL. When this list contains targets, this integration instance collects
  from exactly those targets and doesn't use discovery. When the list is empty or omitted, the
  integration discovers Elasticsearch targets from Kubernetes pod labels and discovery
  annotations. Configure a second named integration instance to use both static targets and
  discovery.

* **`endpoints[].endpoint`**
  Type: `string`. Required.
  Elasticsearch target as either `host:port` for an HTTP connection or a full HTTP or HTTPS URL.

* **`username`**
  Type: `string`. Optional.
  Username for HTTP basic authentication to statically configured targets. Set `username` and
  `password` together. Targets discovered through discovery annotations use the credentials in
  their annotations.

* **`password`**
  Type: `string`. Optional.
  Password for HTTP basic authentication to statically configured targets. Set `username` and
  `password` together. The Collector masks this value in diagnostic output, logs, and errors.
  Targets discovered through discovery annotations use the credentials in their annotations.

* **`collection_interval`**
  Type: `duration`. Optional. Default: `15s`.
  How often the Collector collects metrics from each Elasticsearch target.

* **`timeout`**
  Type: `duration`. Optional. Default: `10s`.
  Maximum time allowed to collect metrics from one Elasticsearch target. The value must not
  exceed `collection_interval`.

* **`nodes`**
  Type: `array of string`. Optional. Default: `["_local"]`.
  Elasticsearch node selectors to collect. Use `_local` to collect the node that receives the
  request, `_all` to collect every node in the cluster, or Elasticsearch node names and selector
  expressions. An empty list disables node metrics.

* **`skip_cluster_metrics`**
  Type: `boolean`. Optional. Default: `false`.
  If true, don't collect cluster metrics, including pending task and index template metrics. If
  false, collect cluster metrics.

* **`indices`**
  Type: `array of string`. Optional. Default: `[]`.
  Elasticsearch index names and index selector expressions to collect. Use `_all` to collect
  every index. An empty list disables index metrics.

* **`pending_tasks`**
  Type: `boolean`. Optional. Default: `true`.
  If true, collect the number, priority, and average queue time of pending cluster tasks. If
  false, don't request pending task details or emit pending task metrics.

* **`tls`**
  Type: `object`. Optional.
  Transport Layer Security (TLS) settings for statically configured HTTPS targets. The Collector
  verifies server certificates by default. Targets discovered through annotations use the TLS
  settings in their annotations.

* **`tls.ca_file`**
  Type: `string`. Optional.
  Path to a PEM-encoded certificate authority (CA) certificate file used to verify the
  Elasticsearch server certificate. If omitted, the Collector uses the system certificate
  authority pool.

* **`tls.ca_pem`**
  Type: `string`. Optional.
  PEM-encoded certificate authority certificate content used to verify the Elasticsearch server
  certificate.

* **`tls.cert_file`**
  Type: `string`. Optional.
  Path to the PEM-encoded client certificate file to present when the Elasticsearch endpoint
  requires mutual TLS authentication.

* **`tls.cert_pem`**
  Type: `string`. Optional.
  PEM-encoded client certificate content to present when the Elasticsearch endpoint requires
  mutual TLS authentication.

* **`tls.cipher_suites`**
  Type: `array of string`. Optional.
  TLS cipher suites that the Collector can use, in preference order. If omitted, the Collector
  uses the secure defaults from the Go TLS library. For supported names, see
  [https://go.dev/src/crypto/tls/cipher\_suites.go](https://go.dev/src/crypto/tls/cipher_suites.go).

* **`tls.curve_preferences`**
  Type: `array of string`. Optional.
  Elliptic curves that the Collector can use for an elliptic curve Diffie-Hellman ephemeral
  handshake, in preference order. If omitted, the Collector uses the Go TLS library defaults.

* **`tls.include_system_ca_certs_pool`**
  Type: `boolean`. Optional.
  Whether to add certificates from the system certificate authority pool to the configured
  certificate authorities. If true, the Collector trusts both sources. If false, the Collector
  trusts only the configured certificate authorities.

* **`tls.insecure`**
  Type: `boolean`. Optional.
  Whether to disable transport security. If true, the Collector uses an unencrypted connection.
  If false, the Collector uses TLS for HTTPS endpoints.

* **`tls.insecure_skip_verify`**
  Type: `boolean`. Optional.
  Whether to skip verification of the Elasticsearch server certificate. If true, the connection
  is encrypted but isn't authenticated and is vulnerable to man-in-the-middle attacks. If false,
  the Collector verifies the certificate.

* **`tls.key_file`**
  Type: `string`. Optional.
  Path to the PEM-encoded client private key file used with `cert_file` for mutual TLS
  authentication.

* **`tls.key_pem`**
  Type: `string`. Optional.
  PEM-encoded client private key content used with `cert_pem` for mutual TLS authentication.

* **`tls.max_version`**
  Type: `string`. Optional.
  Maximum TLS protocol version the Collector can use. If omitted, the Collector uses the maximum
  version supported by the Go TLS library.

* **`tls.min_version`**
  Type: `string`. Optional.
  Minimum TLS protocol version the Collector can use. If omitted, the minimum version is TLS
  1.2.

* **`tls.reload_interval`**
  Type: `duration`. Optional.
  How often the Collector reloads certificate and key files. If omitted, the Collector doesn't
  reload the files.

* **`tls.server_name_override`**
  Type: `string`. Optional.
  Server name to verify in the Elasticsearch certificate instead of the endpoint hostname. Use
  this setting when the certificate name differs from the endpoint hostname.

* **`tls.tpm`**
  Type: `object`. Optional.
  Trusted platform module settings for the TLS private key.

* **`tls.tpm.auth`**
  Type: `string`. Optional.
  Authorization value for the trusted platform module key.

* **`tls.tpm.enabled`**
  Type: `boolean`. Optional.
  Whether to use a trusted platform module for the TLS private key. If true, the Collector loads
  the key from the configured device or socket. If false, the Collector uses the configured key
  file or in-memory key.

* **`tls.tpm.owner_auth`**
  Type: `string`. Optional.
  Owner authorization value for the trusted platform module.

* **`tls.tpm.path`**
  Type: `string`. Optional.
  Path to the trusted platform module device or Unix domain socket, such as `/dev/tpm0` or
  `/dev/tpmrm0`.


## Related topics

- [Elasticsearch destination plugin](/ingest/pipeline/plugins/destination-plugins/elasticsearch.md)
- [Elasticsearch HTTP source plugin](/ingest/pipeline/plugins/source-plugins/elasticsearch-http.md)
- [Route logs through Palo Alto Networks Telemetry Pipeline](/ingest/logs/pipeline-logs.md)
- [Send telemetry data with destination plugins](/ingest/pipeline/plugins/destination-plugins.md)
- [Ingest telemetry data with source plugins](/ingest/pipeline/plugins/source-plugins.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.