Elasticsearch

The Elasticsearch integration requires CXDOT Collector 1.7.0 or greater. Elasticsearch is a distributed search and analytics engine. Use the Elasticsearch integration to collect cluster, node, operating system, process, and Java Virtual Machine metrics from Elasticsearch servers running in your environment. The Elasticsearch integration supports Elasticsearch 7.9 or later.

​

Supported telemetry types

The Elasticsearch integration supports these telemetry types:

​

Prerequisites

The Elasticsearch integration has the following prerequisites:

• If Elasticsearch security is enabled, create a monitoring user with the monitor or manage cluster privilege. The monitor privilege provides the required read-only access. For more information, see Elasticsearch security privileges.
• Make the Elasticsearch HTTP API reachable from the CXDOT Collector.

​

Configure

The Elasticsearch integration is enabled by default. To configure the integration, follow these steps:

1. For Elasticsearch pods that expose an unsecured HTTP API on port 9200, add the app.kubernetes.io/name: elasticsearch label to the pod template. The CXDOT Collector also discovers pods that carry autodiscovery annotations for Elasticsearch. Use annotations to supply credentials, an HTTPS URL, or a custom port for each target. Metrics from discovered pods arrive with that pod’s Kubernetes metadata attached. For more information, see autodiscovery.
2. Optional: Configure static targets instead of discovered targets, such as Elasticsearch servers outside your Kubernetes cluster. Supply the password through your deployment’s secret management. For example, add the following to the values.yaml for your CXDOT Collector Helm chart:
   config:
     integrations:
       elasticsearch:
         endpoints:
           - endpoint: https://elasticsearch.example.com:9200
         username: cxdot
         password: "${env:ELASTICSEARCH_PASSWORD}"
   
   When you configure static targets, this integration instance collects from exactly those targets instead of discovered targets.
3. Optional: Configure a second integration instance to collect from both discovered pods and static targets. For example, add the following to the values.yaml for your CXDOT Collector Helm chart:
   config:
     integrations:
       elasticsearch: {}
       elasticsearch/edge:
         endpoints:
           - endpoint: elasticsearch.example.com:9200
   
   The elasticsearch/edge key defines a named instance for the static target. The bare elasticsearch key continues to use autodiscovery.
4. Optional: Disable the integration. For example, add the following to the values.yaml for your CXDOT Collector Helm chart:
   config:
     integrations:
       elasticsearch:
         enabled: false
   

​

Validate

To validate the Elasticsearch integration, follow these steps:

1. In the Live Telemetry Analyzer, filter for __name__=cxdot.integration.target.health and cxdot.integration.name=elasticsearch. Confirm that each reachable Elasticsearch target reports 1, identified by its server.address and server.port attributes.
2. In Metrics Explorer, run the following query after Elasticsearch has indexed documents:
   sum by ("server.address", "server.port") ({"elasticsearch.node.documents", "state"="active"})
   
   Confirm that the query returns the expected document count for each target.

​

Configuration reference

Configure one Elasticsearch integration instance with the following settings. In Helm values, place these settings under config.integrations.elasticsearch. In a Collector configuration file, place them under cxdot.integrations.elasticsearch.

​

Optional settings

• enabled Type: boolean. Optional. Default: true. Whether to enable this Elasticsearch integration instance. If true, the Collector collects Elasticsearch metrics. If false, the Collector doesn’t run this integration instance.
• endpoints Type: array of object. Optional. Default: []. Static Elasticsearch targets. Each endpoint is either host:port for an HTTP connection or a full HTTP or HTTPS URL. When this list contains targets, this integration instance collects from exactly those targets and doesn’t use discovery. When the list is empty or omitted, the integration discovers Elasticsearch targets from Kubernetes pod labels and discovery annotations. Configure a second named integration instance to use both static targets and discovery.
• endpoints[].endpoint Type: string. Required. Elasticsearch target as either host:port for an HTTP connection or a full HTTP or HTTPS URL.
• username Type: string. Optional. Username for HTTP basic authentication to statically configured targets. Set username and password together. Targets discovered through discovery annotations use the credentials in their annotations.
• password Type: string. Optional. Password for HTTP basic authentication to statically configured targets. Set username and password together. The Collector masks this value in diagnostic output, logs, and errors. Targets discovered through discovery annotations use the credentials in their annotations.
• collection_interval Type: duration. Optional. Default: 15s. How often the Collector collects metrics from each Elasticsearch target.
• timeout Type: duration. Optional. Default: 10s. Maximum time allowed to collect metrics from one Elasticsearch target. The value must not exceed collection_interval.
• nodes Type: array of string. Optional. Default: ["_local"]. Elasticsearch node selectors to collect. Use _local to collect the node that receives the request, _all to collect every node in the cluster, or Elasticsearch node names and selector expressions. An empty list disables node metrics.
• skip_cluster_metrics Type: boolean. Optional. Default: false. If true, don’t collect cluster metrics, including pending task and index template metrics. If false, collect cluster metrics.
• indices Type: array of string. Optional. Default: []. Elasticsearch index names and index selector expressions to collect. Use _all to collect every index. An empty list disables index metrics.
• pending_tasks Type: boolean. Optional. Default: true. If true, collect the number, priority, and average queue time of pending cluster tasks. If false, don’t request pending task details or emit pending task metrics.
• tls Type: object. Optional. Transport Layer Security (TLS) settings for statically configured HTTPS targets. The Collector verifies server certificates by default. Targets discovered through annotations use the TLS settings in their annotations.
• tls.ca_file Type: string. Optional. Path to a PEM-encoded certificate authority (CA) certificate file used to verify the Elasticsearch server certificate. If omitted, the Collector uses the system certificate authority pool.
• tls.ca_pem Type: string. Optional. PEM-encoded certificate authority certificate content used to verify the Elasticsearch server certificate.
• tls.cert_file Type: string. Optional. Path to the PEM-encoded client certificate file to present when the Elasticsearch endpoint requires mutual TLS authentication.
• tls.cert_pem Type: string. Optional. PEM-encoded client certificate content to present when the Elasticsearch endpoint requires mutual TLS authentication.
• tls.cipher_suites Type: array of string. Optional. TLS cipher suites that the Collector can use, in preference order. If omitted, the Collector uses the secure defaults from the Go TLS library. For supported names, see https://go.dev/src/crypto/tls/cipher_suites.go.
• tls.curve_preferences Type: array of string. Optional. Elliptic curves that the Collector can use for an elliptic curve Diffie-Hellman ephemeral handshake, in preference order. If omitted, the Collector uses the Go TLS library defaults.
• tls.include_system_ca_certs_pool Type: boolean. Optional. Whether to add certificates from the system certificate authority pool to the configured certificate authorities. If true, the Collector trusts both sources. If false, the Collector trusts only the configured certificate authorities.
• tls.insecure Type: boolean. Optional. Whether to disable transport security. If true, the Collector uses an unencrypted connection. If false, the Collector uses TLS for HTTPS endpoints.
• tls.insecure_skip_verify Type: boolean. Optional. Whether to skip verification of the Elasticsearch server certificate. If true, the connection is encrypted but isn’t authenticated and is vulnerable to man-in-the-middle attacks. If false, the Collector verifies the certificate.
• tls.key_file Type: string. Optional. Path to the PEM-encoded client private key file used with cert_file for mutual TLS authentication.
• tls.key_pem Type: string. Optional. PEM-encoded client private key content used with cert_pem for mutual TLS authentication.
• tls.max_version Type: string. Optional. Maximum TLS protocol version the Collector can use. If omitted, the Collector uses the maximum version supported by the Go TLS library.
• tls.min_version Type: string. Optional. Minimum TLS protocol version the Collector can use. If omitted, the minimum version is TLS 1.2.
• tls.reload_interval Type: duration. Optional. How often the Collector reloads certificate and key files. If omitted, the Collector doesn’t reload the files.
• tls.server_name_override Type: string. Optional. Server name to verify in the Elasticsearch certificate instead of the endpoint hostname. Use this setting when the certificate name differs from the endpoint hostname.
• tls.tpm Type: object. Optional. Trusted platform module settings for the TLS private key.
• tls.tpm.auth Type: string. Optional. Authorization value for the trusted platform module key.
• tls.tpm.enabled Type: boolean. Optional. Whether to use a trusted platform module for the TLS private key. If true, the Collector loads the key from the configured device or socket. If false, the Collector uses the configured key file or in-memory key.
• tls.tpm.owner_auth Type: string. Optional. Owner authorization value for the trusted platform module.
• tls.tpm.path Type: string. Optional. Path to the trusted platform module device or Unix domain socket, such as /dev/tpm0 or /dev/tpmrm0.