> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Hadoop YARN

> Hadoop YARN cluster, application, node, and CapacityScheduler queue metrics read from a ResourceManager's REST API.

The Hadoop YARN integration requires CXDOT Collector 1.18.0 or greater.

[Apache Hadoop](https://hadoop.apache.org/) is an open source framework for distributed
storage and processing, and its Yet Another Resource Negotiator (YARN) component
schedules the applications that run on a cluster. Use the Hadoop YARN integration with the
CXDOT Collector to collect cluster, application, node, and queue metrics from the
ResourceManagers of Hadoop clusters running in your environment.

The Hadoop YARN integration supports Apache Hadoop 3.4 or greater.

## Supported telemetry types

The Hadoop YARN integration supports these telemetry types:

| Type | Supported |
| - | - |
| Logs | No |
| Metrics | Yes |
| Traces | No |
| Events | No |

## Prerequisites

The Hadoop YARN integration has the following prerequisites:

* Make the ResourceManager's web interface, which listens on port `8088` by default,
  reachable from the CXDOT Collector. The integration reads the ResourceManager's REST
  API, and the web interface can serve it over HTTP or HTTPS. To confirm that the CXDOT
  Collector can read it, run the following command from a pod or host on the same
  network as the CXDOT Collector, replacing `RESOURCEMANAGER_ADDRESS` with the
  ResourceManager's address. The command returns the HTTP status code `200`. On a
  cluster with ResourceManager high availability, a standby ResourceManager returns
  `307` instead, which is expected:

  ```shell theme={null}
  curl -s -o /dev/null -w '%{http_code}\n' http://RESOURCEMANAGER_ADDRESS:8088/ws/v1/cluster/metrics
  ```

* Check how the ResourceManager's web interface authenticates requests. The integration
  supports unauthenticated access, HTTP basic authentication, custom request headers
  such as a bearer token for an authenticating proxy, and Transport Layer Security (TLS).
  The integration doesn't support Kerberos (SPNEGO) authentication, so a web interface
  that requires Kerberos returns an error for every request.

* On a cluster with ResourceManager high availability, choose how to target the
  ResourceManagers. A standby ResourceManager redirects every request to the active one
  on another host. Use one of the following:

  * Annotate the ResourceManager pods, or list each ResourceManager as a static target.
    The integration treats a target that redirects to another host as a standby. The
    active ResourceManager reports the cluster's metrics, and each standby reports as
    healthy and reports no other metrics, so the cluster's metrics aren't duplicated.
  * Annotate a Kubernetes Service that selects the ResourceManager pods. The integration
    follows the standby's redirect for a target it discovers from a Service, so the
    target reports the cluster's metrics whichever ResourceManager answers.
  * List one address that reaches both ResourceManagers, such as a virtual IP address or
    a load balancer, as a static target, and set `follow_standby_redirects` to `true`.
    Without that setting, the target reports as healthy and reports no metrics whenever
    one of its requests reaches the standby. You don't need the setting if the address
    routes requests only to the active ResourceManager. For example, a load balancer
    can run a health check on the ResourceManager's `/isActive` path, which returns the
    HTTP status code `200` only on the active ResourceManager.

  When the integration follows a standby's redirect, it sends the `headers` values to
  the active ResourceManager but not the `username` and `password` credentials.

## Configure

The Hadoop YARN integration is enabled by default, but it collects only from the
ResourceManagers you give it. You can configure targets either by annotating
ResourceManager pods in your Kubernetes cluster or by listing static endpoints in your
Helm values. To configure the integration, follow these steps:

1. To collect from a ResourceManager running in Kubernetes, add a CXDOT integration
   annotation to the pod template. The annotation key must end with the name of the
   ResourceManager's container, which is `resourcemanager` in the following example. The
   `yarn:` block in the annotation's value chooses the Hadoop YARN integration. For
   example, add the following annotation to your ResourceManager workload:

   ```yaml theme={null}
   spec:
     template:
       metadata:
         annotations:
           integrations.cxdot.paloaltonetworks.com/resourcemanager: |
             schema_version: v1.18
             yarn:
               endpoints:
                 - endpoint: ${target.host}:${target.port.ui}
                   cluster_name: analytics
       spec:
         containers:
           - name: resourcemanager
             image: example.com/hadoop:3.5.0
             ports:
               - name: ui
                 containerPort: 8088
   ```

   The Collector replaces `${target.host}` with the pod IP address and `${target.port.ui}`
   with the container port named `ui`, and it reads the ResourceManager over HTTP. The
   `cluster_name` value becomes the `yarn.cluster.name` attribute on the metrics, which
   tells clusters apart because a ResourceManager doesn't report its cluster's name.

   On a cluster with ResourceManager high availability, an annotation on the pod template
   reaches every ResourceManager. The active ResourceManager reports the cluster's
   metrics. Each standby ResourceManager reports `cxdot.integration.target.health` as `1`
   and no other metrics.

   To collect through a Service that selects the ResourceManager pods instead, add the
   same annotation value to the Service under the
   `integrations.cxdot.paloaltonetworks.com/service` key. The Collector then replaces
   `${target.host}` with the Service's cluster IP address and `${target.port.ui}` with
   the Service port named `ui`, and it follows a standby ResourceManager's redirect to
   the active one.

2. Optional: To collect from ResourceManagers outside your Kubernetes cluster, configure
   static targets. For example, add the following to the Helm values file for your CXDOT
   Collector:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         yarn:
           endpoints:
             - endpoint: resourcemanager-a.example.com:8088
               cluster_name: analytics
             - endpoint: https://resourcemanager-b.example.com:8090
               cluster_name: reporting
   ```

   Write a target as `host:port` to read it over HTTP, or as a full HTTP or HTTPS URL.
   When you configure static targets, this integration instance collects from exactly
   those targets instead of discovered targets. In cluster mode, the cluster Collector
   that wins leader election scrapes static targets.

3. Optional: Configure a second integration instance to collect from both annotated
   pods and static targets. For example, add the following to the Helm values file for
   your CXDOT Collector:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         yarn: {}
         yarn/external:
           endpoints:
             - endpoint: resourcemanager-a.example.com:8088
               cluster_name: analytics
   ```

   The `yarn/external` key defines a named instance for the static targets. The bare
   `yarn` key continues to collect from annotated pods.

4. Optional: If the ResourceManager's web interface requires HTTP basic authentication
   or HTTPS, configure the matching settings for static targets, and provide the password
   through your deployment's secret management. For example, the following Helm values
   reference the `YARN_PASSWORD` environment variable and a mounted certificate authority
   file:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         yarn:
           endpoints:
             - endpoint: https://resourcemanager-a.example.com:8090
               cluster_name: analytics
           username: cxdot
           password: "${env:YARN_PASSWORD}"
           tls:
             ca_file: /etc/cxdot/yarn-tls/ca.crt
   ```

   The `username`, `password`, and `tls` settings in your Helm values apply to the
   targets you list there. Set `username` and `password` together. You can combine
   them with `headers`. If `headers` also sets an `Authorization` header, the
   integration sends the basic credentials in its place.

   The integration follows an HTTP redirect only when the redirect stays on the target's
   host, for example from a proxy to another path on the proxy. To make such a request fail
   instead, set `follow_redirects` to `false`. The integration follows a redirect to
   another host only for a target it discovers from a Service, or when
   `follow_standby_redirects` is `true`, as described in Prerequisites. Otherwise, it
   sends the credentials and the `headers` values only to the host you configured.

   Set the password variable on both the node and cluster Collectors with the
   `nodeCollector.extraEnv` and `clusterCollector.extraEnv` Helm values. Mount the
   certificate authority file into the cluster Collector, which collects from static
   targets, with the `clusterCollector.extraVolumes` and
   `clusterCollector.extraVolumeMounts` Helm values.

   For annotated pods, set these settings in the CXDOT integration annotation instead.
   Reference the password with `password: ${k8s_secret:yarn-credentials/password}` and
   supply the certificate authority from a Kubernetes Secret with
   `tls.ca_pem: ${k8s_secret:yarn-credentials/ca.crt}`. To allow the Collector to read
   the Secret, add the pod's namespace to `apiServer.metadataServer.secretAllowlist` in
   the Helm values for your CXDOT Collector. Use a URL that starts with `https://` in the
   annotation's `endpoint` value. Because `${target.host}` is the pod IP address, set
   `tls.server_name_override` to a name that the ResourceManager's certificate carries,
   unless the certificate carries the pod IP address.

5. Optional: To limit the series the application, node, and queue metrics create, adjust
   which applications, nodes, and queues report. For example, add the following to the Helm
   values file for your CXDOT Collector:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         yarn:
           endpoints:
             - endpoint: resourcemanager-a.example.com:8088
               cluster_name: analytics
           application_states: [RUNNING, ACCEPTED]
           max_applications: 500
           max_nodes: 500
           inactive_node_retention: 1h
           exclude_queues: [adhoc]
           max_queues: 50
   ```

   These settings in your Helm values apply to the static targets of the same instance.
   For annotated pods, set `application_states`, `application_id`, `max_applications`,
   `node_metrics`, `max_nodes`, `inactive_node_retention`, `queue_metrics`,
   `exclude_queues`, and `max_queues` in the CXDOT integration annotation instead.

   The integration reports `yarn.application.status` for every application that hasn't
   ended, and for every failed or killed application that the ResourceManager still
   lists. The status is `2`, which means critical, when the application's state or its
   final status is `FAILED` or `KILLED`. An application whose application master reports
   a failure and then exits cleanly, such as a Spark application whose driver fails,
   ends in the `FINISHED` state with the final status `FAILED`. The integration also
   reports a critical status for those applications, so an alert on a critical status
   covers failed and killed applications without changing a setting.

   The `application_states` setting chooses which applications also report resource and
   progress metrics, and defaults to `RUNNING`. Other finished applications report a
   status only if you add `FINISHED` to `application_states`. That includes a finished
   application whose final status is `KILLED`, which then reports a critical status.
   The `yarn.cluster.application.terminations` metric counts ended applications by
   final state without this setting.

   By default, every `yarn.application.*` series carries the application's ID in the
   `yarn.application.id` attribute, so each run of an application creates a new set of
   series. On a cluster that runs many short-lived applications, set `application_id` to
   `false` to leave the ID off. Applications that share a name, queue, state, and user
   then report as one, with their memory, virtual cores, and containers summed. The
   integration also stops reporting `yarn.application.memory.consumed` and
   `yarn.application.vcore.consumed`.

   The `max_applications` limit, which defaults to `2000`, caps how many applications
   report, or how many groups of applications when `application_id` is `false`.
   Applications that haven't ended report first, starting with the one that has run
   longest, so an application that reports keeps reporting while newer ones arrive.
   Ended applications follow, starting with the one that ended most recently. If the
   applications that haven't ended fill the limit, failed and killed applications report
   no `yarn.application.status`.

   A ResourceManager keeps listing a node that has left the cluster until the
   ResourceManager restarts. A NodeManager that listens on an ephemeral port, which is
   the default, registers as a new node each time it starts, so the listed nodes grow
   with every NodeManager restart. The integration reports `yarn.node.*` metrics for a
   node whose state is `LOST`, `SHUTDOWN`, `DECOMMISSIONED`, or `REBOOTED` only while
   the node's last health report is newer than `inactive_node_retention`, which defaults
   to `4h`. The `max_nodes` limit, which defaults to `1000`, caps how many nodes report,
   and nodes that are part of the cluster report first. The `yarn.cluster.node.count`
   metric counts every node by state regardless of both settings.

   The integration fills the `max_queues` limit, which defaults to `100`, from the queue
   nearest the root outward, one level at a time, and each level follows the order that
   the scheduler lists its queues in. On a scheduler with more queues, the deepest
   queues stop reporting first. Use `exclude_queues` to choose which queues report, by
   name or by full path. A queue you exclude also excludes its child queues. On a
   cluster that uses the Fair scheduler or the first in, first out (FIFO) scheduler,
   which report no queue metrics, set `queue_metrics` to `false` so the integration
   doesn't request scheduler information.

6. Optional: Disable the integration. For example, add the following to the Helm values
   file for your CXDOT Collector:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         yarn:
           enabled: false
   ```

Metrics from each discovered pod include that pod's Kubernetes metadata. For more
information, see
[autodiscovery](https://docs-xcor.paloaltonetworks.com/ingest/xcor/collector/discover/kubernetes.md).

### Validate

To validate the Hadoop YARN integration, follow these steps:

1. In [Live Telemetry Analyzer](https://docs-xcor.paloaltonetworks.com/investigate/analyze/telemetry-analyzer.md),
   filter for `__name__=cxdot.integration.target.health` and
   `cxdot.integration.name=yarn`. Confirm that `cxdot.integration.target.health` reports
   `1` for each ResourceManager, including each standby ResourceManager. A value of `0`
   means that at least one request from the CXDOT Collector to that ResourceManager
   failed. The `cxdot.integration.target`
   attribute identifies the target.

2. In [Metrics Explorer](https://docs-xcor.paloaltonetworks.com/investigate/querying/metrics/explorer.md),
   run the following query:

   ```text theme={null}
   sum by ("server.address", "server.port", "yarn.node.state") ({"yarn.cluster.node.count"})
   ```

   Confirm that the query returns a time series for each node state of each active
   ResourceManager, and that the `running` series reports the number of NodeManagers
   that are running. The integration reports memory in MiB and times in milliseconds, as
   the ResourceManager reports them.

### Troubleshooting

* `cxdot.integration.target.health` reads `0`, but some metrics still report: The
  CXDOT Collector makes up to five requests to the ResourceManager, for cluster,
  application, node, and queue data. A request that fails or times out costs its own
  metrics, and the others still report. Check which metric prefix is missing: `yarn.cluster.*`,
  `yarn.application.*`, `yarn.node.*`, or `yarn.queue.*`.
* Every request fails for a secured cluster: The integration doesn't support Kerberos
  (SPNEGO) authentication. Reach a ResourceManager that requires it through an address
  that accepts basic authentication or a header-based proxy, and set `username` and
  `password` or `headers` to match.
* No `yarn.queue.*` metrics appear: The integration reads queues only from the
  CapacityScheduler. With the Fair scheduler or the first in, first out scheduler, the
  integration still reports the `yarn.cluster.*`, `yarn.application.*`, and
  `yarn.node.*` metrics.
  The `queue_metrics` setting also turns these metrics off.
* Some queues report no metrics: The scheduler has more queues than `max_queues`, and the
  Collector logs a warning that says so. Raise `max_queues` or use `exclude_queues`.
* Some applications report no metrics, or failed and killed applications report no
  `yarn.application.status`: The ResourceManager lists more applications than
  `max_applications`, and the Collector logs a warning that says so. Applications that
  haven't ended fill the limit first. Raise `max_applications`, or set `application_id`
  to `false` so that applications that share a name, queue, state, and user count
  once.
* `cxdot.integration.target.health` reads `1`, but no `yarn.*` metrics report: The
  target redirects to another host, so the integration treats it as a standby
  ResourceManager, and the Collector logs a message that names the host. For a standby
  ResourceManager, the active ResourceManager's target reports the metrics. For a
  target that's one address for both ResourceManagers, such as a virtual IP address
  or a load balancer, a collection reports no metrics whenever one of its requests
  reaches the standby. Set `follow_standby_redirects` to `true` for that target, or
  route its requests only to the active ResourceManager, as described in Prerequisites.
  For any other target, such as a proxy that redirects to another host name, target
  the address it redirects to.
* Every request fails with `without a usable Location`: A standby ResourceManager
  answers that way while no ResourceManager is active. Check the state of the
  ResourceManagers.
* Some nodes report no `yarn.node.*` metrics: The ResourceManager lists more nodes than
  `max_nodes`, and the Collector logs a warning that says so. Nodes that are part of
  the cluster fill the limit first. Raise `max_nodes`. To stop collecting the
  `yarn.node.*` metrics, set `node_metrics` to `false`. The `yarn.cluster.*` metrics
  still report the cluster's totals and its node counts by state.
* A lost, shut down, decommissioned, or rebooted node stops reporting `yarn.node.*`
  metrics: The node's last health report is older than `inactive_node_retention`. A
  NodeManager reports its health when its health checks run. If the NodeManager runs
  neither the disk health check nor a health script, its last health report is the time
  it started, so a node that ran longer than `inactive_node_retention` stops reporting
  as soon as it leaves the cluster. Raise `inactive_node_retention` to keep such nodes
  reporting longer. The `yarn.cluster.node.count` metric still counts the node.
* A redirected request fails with `redirect not followed`: The target redirects to
  another address on the same host, and `follow_redirects` is `false` or your deployment
  denies redirects for annotated pods. Target the address the ResourceManager redirects
  to.
* `yarn.application.memory.consumed` and `yarn.application.vcore.consumed` are missing:
  The integration doesn't report them when `application_id` is `false`.
* Metrics for a Spark application's own executors and jobs don't appear: The integration
  reads only the ResourceManager. The Spark integration collects those metrics from the
  Spark driver.

## Configuration reference

Configure one Hadoop YARN integration instance with the following settings. In Helm values,
place these settings under `collection.cxdot.integrations.yarn`. In a Collector configuration
file, place them under `cxdot.integrations.yarn`.

### Optional settings

* **`enabled`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to enable this Hadoop YARN integration instance. If true, the Collector runs the
  instance. If false, the Collector doesn't run it.

* **`endpoints`**
  Type: `array of object`. Optional. Default: `[]`.
  ResourceManager targets. Specify a target as `host:port` to use HTTP, or as a full HTTP or
  HTTPS URL of the ResourceManager's web interface, without a REST API path. On a cluster with
  ResourceManager high availability, list each ResourceManager. A standby ResourceManager
  redirects to the active one, and the Collector treats a target that redirects to another host
  as a standby: the target reports as healthy and reports no metrics. If you list one address
  that reaches both ResourceManagers instead, such as a virtual IP address or a load balancer,
  set `follow_standby_redirects` to `true`, unless the address routes requests only to the
  active ResourceManager.

* **`endpoints[].endpoint`**
  Type: `string`. Required.
  `host:port` of the ResourceManager's web interface, or its HTTP or HTTPS URL.

* **`endpoints[].cluster_name`**
  Type: `string`. Optional.
  Name of the YARN cluster the ResourceManager belongs to, reported as the `yarn.cluster.name`
  attribute. When omitted, the attribute isn't set.

* **`application_metrics`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to collect the `yarn.application.*` metrics. When `false`, the Collector doesn't
  request the ResourceManager's application list.

* **`application_id`**
  Type: `boolean`. Optional. Default: `true`.
  Whether the `yarn.application.*` metrics carry the application's ID in the
  `yarn.application.id` attribute. When `true`, each run of an application is its own set of
  series. When `false`, the Collector leaves the ID off, and applications that share a name,
  queue, state, and user report as one. Their memory, virtual cores, and containers are summed.
  Progress is the lowest, start time the earliest, and finish time and elapsed time the latest
  and longest. The Collector then doesn't report `yarn.application.memory.consumed` and
  `yarn.application.vcore.consumed`, because a total summed over applications that come and go
  would fall whenever one ends. Set this to `false` on a cluster that runs many short-lived
  applications.

* **`max_applications`**
  Type: `integer`. Optional. Default: `2000`. Minimum: `1`.
  Cap on how many applications report `yarn.application.*` metrics, or, when `application_id` is
  `false`, on how many groups of applications that report as one. Applications that haven't
  ended report first, starting with the one that has run longest, so an application that reports
  keeps reporting while newer ones arrive. Ended applications follow, starting with the one that
  ended most recently. On a ResourceManager with more applications than the cap, the rest report
  no metrics and the Collector logs a warning. If the applications that haven't ended fill the
  cap, failed and killed applications report no `yarn.application.status`.

* **`application_states`**
  Type: `array of string`. Optional. Default: `["RUNNING"]`.
  Application states to collect resource and progress metrics for. This setting doesn't limit
  `yarn.application.status`, which the Collector reports for every application that hasn't ended
  (`NEW`, `NEW_SAVING`, `SUBMITTED`, `ACCEPTED`, and `RUNNING`) and for every failed or killed
  application the ResourceManager still lists. The exception is finished applications, which
  make up most of the ended applications a ResourceManager retains. Unless you list `FINISHED`
  here, the Collector asks the ResourceManager only for the finished applications whose final
  status is `FAILED`, such as a Spark application whose driver reported a failure, and reports
  their status as critical. When you list `FINISHED`, every finished application reports a
  status, which is critical if its final status is `FAILED` or `KILLED`. The Collector reports a
  finished application whose final status is `KILLED` only when you list `FINISHED`. The
  `yarn.cluster.application.terminations` metric counts ended applications by final state.

* **`node_metrics`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to collect the `yarn.node.*` metrics. When `false`, the Collector doesn't request the
  ResourceManager's node list.

* **`max_nodes`**
  Type: `integer`. Optional. Default: `1000`. Minimum: `1`.
  Cap on how many nodes report `yarn.node.*` metrics. Nodes that are part of the cluster report
  first, in order of node ID. Nodes that have left the cluster and are still within
  `inactive_node_retention` follow, starting with the one that reported its health most
  recently. On a ResourceManager with more nodes than the cap, the rest report no `yarn.node.*`
  metrics and the Collector logs a warning. The `yarn.cluster.node.count` metric counts every
  node regardless of this setting.

* **`inactive_node_retention`**
  Type: `duration`. Optional. Default: `4h`.
  How long a node that has left the cluster keeps reporting `yarn.node.*` metrics, measured from
  its last health report. A node has left the cluster when its state is `LOST`, `SHUTDOWN`,
  `DECOMMISSIONED`, or `REBOOTED`. A ResourceManager keeps listing such a node until the
  ResourceManager restarts. A NodeManager that listens on an ephemeral port, which is the
  default, registers as a new node each time it starts, so without this limit the number of
  nodes that report grows with every NodeManager restart. Set this to `0s` to report no nodes
  that have left the cluster. The `yarn.cluster.node.count` metric counts every node regardless
  of this setting.

* **`queue_metrics`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to collect the `yarn.queue.*` metrics. When `false`, the Collector doesn't request the
  ResourceManager's scheduler information, and the `yarn.queue.path` attribute of the
  `yarn.application.*` metrics is the queue as the ResourceManager reports it for the
  application. Set this to `false` on a cluster that uses the Fair scheduler or the first in,
  first out scheduler, which report no queue metrics.

* **`exclude_queues`**
  Type: `array of string`. Optional. Default: `[]`.
  CapacityScheduler queues to leave out, by name (`adhoc`) or by full path
  (`root.analytics.adhoc`). Leaving a queue out also leaves out its child queues.

* **`max_queues`**
  Type: `integer`. Optional. Default: `100`. Minimum: `1`.
  Cap on how many queues, not counting the root, report metrics, filled from the queue nearest
  the root outward, one level at a time, with each level in the order the scheduler lists its
  queues. On a scheduler with more queues, use `exclude_queues` to choose which ones report.

* **`username`**
  Type: `string`. Optional.
  Username for HTTP basic authentication to the targets in `endpoints`. Set `username` and
  `password` together. The Collector sends the credentials in an `Authorization` header on every
  request, in place of an `Authorization` header set in `headers`. Kerberos (SPNEGO)
  authentication isn't supported.

* **`password`**
  Type: `string`. Optional.
  Password for HTTP basic authentication to the targets in `endpoints`. Set `username` and
  `password` together. The Collector masks this value in diagnostic output, logs, and errors.
  Kerberos (SPNEGO) authentication isn't supported.

* **`headers`**
  Type: `object`. Optional.
  Additional request headers as name-value pairs, sent to every target in `endpoints`, for
  example a bearer token in `Authorization` for a ResourceManager behind an authenticating
  proxy. You can combine `headers` with `username` and `password`. If you do, the Collector
  doesn't send an `Authorization` header set here, and sends the basic credentials instead. The
  Collector sends these headers only to the target's own host, unless `follow_standby_redirects`
  is `true`. In that case, the Collector also sends them to the active ResourceManager that a
  standby redirects to. The Collector masks the values in diagnostic output, logs, and errors.
  This setting doesn't enable Kerberos (SPNEGO) authentication, which isn't supported.

* **`follow_redirects`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to follow HTTP redirects that stay on the target's host, for example from a proxy to
  another path on the proxy. If false, such a redirect causes the metric collection to fail.
  Redirects to another host are controlled by `follow_standby_redirects`, and the Collector
  follows them only if both settings are true.

* **`follow_standby_redirects`**
  Type: `boolean`. Optional. Default: `false`.
  Whether to follow a standby ResourceManager's redirect to the active ResourceManager. A
  standby ResourceManager answers every request with a redirect to the active one on another
  host. If false, the Collector doesn't follow that redirect and treats the target as a standby:
  the target reports as healthy and reports no metrics. Use false when each target is a single
  ResourceManager, so that only the active one reports the cluster's metrics. If true, the
  Collector follows the redirect, and the target reports the cluster's metrics whichever
  ResourceManager answers. Use true when a target is one address that reaches both
  ResourceManagers, such as a virtual IP address or a load balancer that doesn't route only to
  the active ResourceManager. With false, such a target reports no metrics whenever one of its
  requests reaches the standby. The Collector always follows the redirect for a target
  discovered from a Kubernetes Service, which is one address for every ResourceManager behind
  it. A redirect to another host is followed only if `follow_redirects` is also true. The
  Collector doesn't send the `username` and `password` credentials to the active ResourceManager
  that a standby redirects to, so if the active ResourceManager requires authentication,
  authenticate with `headers`.

* **`collection_interval`**
  Type: `duration`. Optional. Default: `15s`.
  How often the Collector collects metrics from each ResourceManager.

* **`timeout`**
  Type: `duration`. Optional. Default: `15s`.
  Maximum time allowed for one collection from a ResourceManager. The limit applies to each
  request the Collector makes and to the collection as a whole. The value must not exceed
  `collection_interval`.

* **`tls`**
  Type: `object`. Optional.
  Transport layer security (TLS) settings for the HTTPS targets in `endpoints`. Certificate
  verification is enabled unless `insecure_skip_verify` is true. These settings don't apply to
  HTTP targets.

* **`tls.ca_file`**
  Type: `string`. Optional.
  Path to a PEM-encoded certificate authority (CA) certificate file used to verify the
  ResourceManager's certificate. If omitted, the Collector uses the system certificate authority
  pool.

* **`tls.ca_pem`**
  Type: `string`. Optional.
  PEM-encoded certificate authority certificate content used to verify the ResourceManager's
  certificate.

* **`tls.cert_file`**
  Type: `string`. Optional.
  Path to the PEM-encoded client certificate file to present when the ResourceManager requires
  mutual TLS authentication.

* **`tls.cert_pem`**
  Type: `string`. Optional.
  PEM-encoded client certificate content to present when the ResourceManager requires mutual TLS
  authentication.

* **`tls.cipher_suites`**
  Type: `array of string`. Optional.
  TLS cipher suites that the Collector can use, in preference order. If omitted, the Collector
  uses secure default cipher suites. For supported names, see
  [https://go.dev/src/crypto/tls/cipher\_suites.go](https://go.dev/src/crypto/tls/cipher_suites.go).

* **`tls.curve_preferences`**
  Type: `array of string`. Optional.
  Elliptic curves that the Collector can use for an elliptic curve Diffie-Hellman ephemeral
  handshake, in preference order. If omitted, the Collector uses its default curves.

* **`tls.include_system_ca_certs_pool`**
  Type: `boolean`. Optional.
  Whether to add certificates from the system certificate authority pool to the configured
  certificate authorities. If true, the Collector trusts both sources. If false, the Collector
  trusts only the configured certificate authorities.

* **`tls.insecure`**
  Type: `boolean`. Optional.
  Whether to ignore the other TLS settings. Leave this setting false. If true and neither
  `ca_file` nor `ca_pem` is set, the Collector ignores every other `tls` setting, including
  `insecure_skip_verify`, and verifies HTTPS targets against the system certificate authority
  pool. To skip certificate verification, set `insecure_skip_verify` instead.

* **`tls.insecure_skip_verify`**
  Type: `boolean`. Optional.
  Whether to skip verification of the ResourceManager's certificate. If true, the connection is
  encrypted but isn't authenticated and is vulnerable to man-in-the-middle attacks. If false,
  the Collector verifies the certificate.

* **`tls.key_file`**
  Type: `string`. Optional.
  Path to the PEM-encoded client private key file used with `cert_file` for mutual TLS
  authentication.

* **`tls.key_pem`**
  Type: `string`. Optional.
  PEM-encoded client private key content used with `cert_pem` for mutual TLS authentication.

* **`tls.max_version`**
  Type: `string`. Optional.
  Maximum TLS protocol version the Collector can use. If omitted, the Collector uses the highest
  TLS version it supports.

* **`tls.min_version`**
  Type: `string`. Optional.
  Minimum TLS protocol version the Collector can use. If omitted, the minimum version is TLS
  1.2.

* **`tls.reload_interval`**
  Type: `duration`. Optional.
  How often the Collector reloads certificate and key files. If omitted, the Collector doesn't
  reload the files.

* **`tls.server_name_override`**
  Type: `string`. Optional.
  Server name to verify in the ResourceManager's certificate instead of the endpoint hostname.
  Use this setting when the certificate name differs from the endpoint hostname.

* **`tls.tpm`**
  Type: `object`. Optional.
  Trusted platform module settings for the TLS private key.

* **`tls.tpm.auth`**
  Type: `string`. Optional.
  Authorization value for the trusted platform module key.

* **`tls.tpm.enabled`**
  Type: `boolean`. Optional.
  Whether to use a trusted platform module for the TLS private key. If true, the Collector loads
  the key from the configured device or socket. If false, the Collector uses the configured key
  file or in-memory key.

* **`tls.tpm.owner_auth`**
  Type: `string`. Optional.
  Owner authorization value for the trusted platform module.

* **`tls.tpm.path`**
  Type: `string`. Optional.
  Path to the trusted platform module device or Unix domain socket, such as `/dev/tpm0` or
  `/dev/tpmrm0`.


## Related topics

- [CXDOT Collector integrations](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector.md)
- [Apache Spark](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/spark.md)
- [Apache HBase](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/hbase.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.