> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Apache HBase

> Apache HBase master and RegionServer metrics collected over JMX, including requests and latency, regions and store files, the block cache, flushes and compactions, the write-ahead log, and the Java Virtual Machine runtime.

The Apache HBase integration requires CXDOT Collector 1.17.0 or greater.

[Apache HBase](https://hbase.apache.org/) is an open source distributed database that
stores large tables on Apache Hadoop. Use the Apache HBase integration with the CXDOT
Collector to collect master, RegionServer, and Java Virtual Machine (JVM) runtime metrics
from HBase clusters running in your environment.

The Apache HBase integration supports Apache HBase 2.5 or greater.

## Supported telemetry types

The Apache HBase integration supports these telemetry types:

| Type | Supported |
| - | - |
| Logs | No |
| Metrics | Yes |
| Traces | No |
| Events | No |

## Prerequisites

The Apache HBase integration has the following prerequisites:

* Enable remote Java Management Extensions (JMX) monitoring on the master and on each
  RegionServer. HBase disables remote JMX by default. Each role runs its own JVM, so set
  the `com.sun.management.jmxremote` system properties in `HBASE_MASTER_OPTS` and in
  `HBASE_REGIONSERVER_OPTS` in `hbase-env.sh`. The commented example in `hbase-env.sh`
  uses port `10101` for the master and port `10102` for the RegionServer. For example,
  the following lines enable remote JMX on those ports without authentication or
  Transport Layer Security (TLS):

  ```shell theme={null}
  export HBASE_JMX_BASE="-Dcom.sun.management.jmxremote \
    -Dcom.sun.management.jmxremote.authenticate=false \
    -Dcom.sun.management.jmxremote.ssl=false \
    -Djava.rmi.server.hostname=SERVER_ADDRESS"
  export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS $HBASE_JMX_BASE \
    -Dcom.sun.management.jmxremote.port=10101 \
    -Dcom.sun.management.jmxremote.rmi.port=10101"
  export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE \
    -Dcom.sun.management.jmxremote.port=10102 \
    -Dcom.sun.management.jmxremote.rmi.port=10102"
  ```

  Replace `SERVER_ADDRESS` with the server's address as the CXDOT Collector reaches it.
  Setting `com.sun.management.jmxremote.rmi.port` to the same value as
  `com.sun.management.jmxremote.port` serves the JMX registry and the connection it
  returns on one port, so each process needs one open port for JMX. For more
  information, see [JMX](https://hbase.apache.org/docs/configuration/important#jmx) in
  the HBase documentation.
* Make the JMX port and the address each process advertises for Remote Method
  Invocation (RMI) connections reachable from the CXDOT Collector. Set
  `java.rmi.server.hostname` to an address the CXDOT Collector can resolve, or
  configure the integration's advertised host and port overrides when that address isn't
  reachable.
* Restrict JMX access to trusted clients. In production, enable JMX password
  authentication and TLS on the master and RegionServers, and configure the
  integration's `username`, `password`, `registry_tls`, and `tls` settings to match. The
  JMX section of the HBase documentation shows the `HBASE_JMX_BASE` settings for both.

## Configure

The Apache HBase integration is enabled by default. You can configure targets either by
annotating master and RegionServer pods in your Kubernetes cluster or by listing static
endpoints in your Helm values. The integration collects from masters and RegionServers
separately: a CXDOT integration annotation or a static endpoint list names the role it
configures, and the metrics from each target carry `hbase.role` set to `master` or
`regionserver`. To configure the integration, follow these steps:

1. To collect from HBase pods in Kubernetes, add a CXDOT integration annotation to the
   pod template of the master workload and to the pod template of the RegionServer
   workload. The annotation key must end with the container name, its value names the
   role under `hbase`, and it can reference JMX credentials stored in a Kubernetes
   Secret in the same namespace. For example, add the following annotation to your
   HBase master workload:

   ```yaml theme={null}
   spec:
     template:
       metadata:
         annotations:
           integrations.cxdot.paloaltonetworks.com/master: |
             schema_version: v1.17
             hbase:
               master:
                 endpoints:
                   - endpoint: ${target.host}:${target.port.jmx}
               username: cxdot
               password: ${k8s_secret:hbase-jmx/password}
       spec:
         containers:
           - name: master
             image: example.com/hbase:2.6.7
             ports:
               - name: jmx
                 containerPort: 10101
   ```

   Add the following annotation to your RegionServer workload:

   ```yaml theme={null}
   spec:
     template:
       metadata:
         annotations:
           integrations.cxdot.paloaltonetworks.com/regionserver: |
             schema_version: v1.17
             hbase:
               regionserver:
                 endpoints:
                   - endpoint: ${target.host}:${target.port.jmx}
               username: cxdot
               password: ${k8s_secret:hbase-jmx/password}
       spec:
         containers:
           - name: regionserver
             image: example.com/hbase:2.6.7
             ports:
               - name: jmx
                 containerPort: 10102
   ```

   The Collector replaces `${target.host}` with the pod IP address, `${target.port.jmx}`
   with the container port named `jmx`, and `${k8s_secret:hbase-jmx/password}` with the
   `password` key of the `hbase-jmx` Secret. Each master and RegionServer pod becomes its
   own target. To allow the Collector to read the Secret, add the pod's namespace to
   `apiServer.metadataServer.secretAllowlist` in the Helm values for your CXDOT
   Collector.

2. Optional: To collect from HBase servers outside your Kubernetes cluster, configure
   static targets for each role. For example, add the following to the Helm values file
   for your CXDOT Collector:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         hbase:
           master:
             endpoints:
               - endpoint: hbase-master.example.com:10101
           regionserver:
             endpoints:
               - endpoint: hbase-rs-0.example.com:10102
               - endpoint: hbase-rs-1.example.com:10102
   ```

   When you configure static targets for a role, this integration instance collects from
   exactly those targets for that role instead of discovered targets. Each role's list
   stands on its own: listing masters under `master.endpoints` leaves RegionServer
   discovery on. In cluster mode, the cluster Collector that wins leader election
   scrapes static targets.

3. Optional: Configure a second integration instance to collect from both annotated
   pods and static targets. For example, add the following to the Helm values file for
   your CXDOT Collector:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         hbase: {}
         hbase/external:
           master:
             endpoints:
               - endpoint: hbase-master.example.com:10101
           regionserver:
             endpoints:
               - endpoint: hbase-rs-0.example.com:10102
   ```

   The `hbase/external` key defines a named instance for the static targets. The bare
   `hbase` key continues to discover pods.

4. Optional: If the master and RegionServers require JMX authentication or TLS,
   configure the matching settings for static targets, and provide the password through
   your deployment's secret management. For example, the following Helm values reference
   the `HBASE_JMX_PASSWORD` environment variable and a mounted certificate authority
   file:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         hbase:
           master:
             endpoints:
               - endpoint: hbase-master.example.com:10101
           regionserver:
             endpoints:
               - endpoint: hbase-rs-0.example.com:10102
           username: cxdot
           password: "${env:HBASE_JMX_PASSWORD}"
           registry_tls: true
           tls:
             ca_file: /etc/cxdot/hbase-tls/ca.crt
   ```

   The `username`, `password`, `registry_tls`, and `tls` settings apply to every master
   and RegionServer that the instance collects from. Every Collector loads the same
   integration settings and rejects a `username` without a `password`, so set the
   password variable on both the node and cluster Collectors with the
   `nodeCollector.extraEnv` and `clusterCollector.extraEnv` Helm values. Mount the
   certificate authority file into the cluster Collector, which collects from static
   targets, with the `clusterCollector.extraVolumes` and
   `clusterCollector.extraVolumeMounts` Helm values.

   For annotated pods, set these settings in the CXDOT integration annotation instead,
   and supply the certificate authority from a Kubernetes Secret with
   `tls.ca_pem: ${k8s_secret:hbase-tls/ca.crt}`. The Collector rejects an annotation
   that sets `tls.ca_file`, or any other setting that names a file, unless the file is
   under
   `collection.cxdot.integrations.discovery.kubernetes.annotation_restrictions.allowed_collector_file_paths`
   in your Helm values, which allows no files by default.

5. Optional: Disable the integration, or one of its roles. For example, add the
   following to the Helm values file for your CXDOT Collector to stop collecting from
   masters and RegionServers alike:

   ```yaml theme={null}
   collection:
     cxdot:
       integrations:
         hbase:
           enabled: false
   ```

   To keep collecting from RegionServers and stop collecting from masters, set
   `master.enabled` to `false` instead.

Metrics from each discovered pod include that pod's Kubernetes metadata. For more
information, see
[autodiscovery](https://docs-xcor.paloaltonetworks.com/ingest/xcor/collector/discover/kubernetes.md).

### Validate

To validate the Apache HBase integration, follow these steps:

1. In [Live Telemetry Analyzer](https://docs-xcor.paloaltonetworks.com/investigate/analyze/telemetry-analyzer.md),
   filter for `__name__=cxdot.integration.target.health` and
   `cxdot.integration.name=hbase`. Confirm that `cxdot.integration.target.health`
   reports `1` for each master and each RegionServer. A value of `0` means the CXDOT
   Collector hit an error connecting to or collecting from that target. The
   `hbase.role`, `cxdot.integration.target`, `server.address`, and `server.port`
   attributes identify the target.

2. In [Metrics Explorer](https://docs-xcor.paloaltonetworks.com/investigate/querying/metrics/explorer.md),
   run the following query:

   ```text theme={null}
   sum by ("server.address", "server.port") ({"hbase.master.region_server.count", "hbase.region_server.state"="live"})
   ```

   Confirm that the query returns one time series per master, each reporting the number
   of RegionServers that master sees as live.

3. In Metrics Explorer, run the following query while the cluster serves reads and
   writes:

   ```text theme={null}
   sum by ("server.address", "server.port", "hbase.request.type") (rate({"hbase.region_server.request.count"}[$__rate_interval]))
   ```

   Confirm that the query returns a `read` and a `write` time series for each
   RegionServer.

### Troubleshooting

* `hbase.wal.*` time series appear for a master: The master keeps a write-ahead log for
  its procedure store and reports it through the same metrics as a RegionServer's log.
  Filter on `hbase.role="regionserver"` to see the logs that record table writes.
* A duration statistic such as `hbase.region_server.operation.duration.p99` reads `0`
  while the cluster serves requests: HBase computes its minimum, maximum, mean, median,
  and p99 statistics over the operations since its last metrics snapshot, which it
  takes every 10 seconds by default, as `*.period` in
  `hadoop-metrics2-hbase.properties` sets. An interval without an operation of that
  kind reads `0`, and an operation faster than a millisecond rounds down to `0`. The
  matching `.count` metrics are cumulative, so query a rate of them for throughput.

For more information about CXDOT Collector integrations, see the
[integrations catalog](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector).

## Configuration reference

Configure one Apache HBase integration instance with the following settings. In Helm values,
place these settings under `collection.cxdot.integrations.hbase`. In a Collector configuration
file, place them under `cxdot.integrations.hbase`.

### Optional settings

* **`enabled`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to enable this Apache HBase integration instance. If true, the Collector collects from
  the masters and RegionServers the instance configures or discovers. If false, the Collector
  doesn't run the instance.

* **`master`**
  Type: `object`. Optional.
  Settings for collecting from HBase masters. Masters are discovered from workload annotations,
  or listed here as endpoints.

* **`master.enabled`**
  Type: `boolean`. Optional. Default: `true`.
  Whether this instance collects from HBase masters. If true, the Collector collects from the
  masters it discovers or the targets in `master.endpoints`. If false, the Collector doesn't
  collect from masters, and RegionServer collection isn't affected.

* **`master.endpoints`**
  Type: `array of object`. Optional. Default: `[]`.
  HBase master Java Management Extensions (JMX) targets. When this list is empty or omitted, the
  integration uses automatic target discovery. When the list contains targets, this integration
  instance collects from exactly those targets and doesn't automatically discover masters.
  Configure a second named instance to use both methods. Each endpoint can be a `host:port`
  Remote Method Invocation (RMI) registry address or a `service:jmx:rmi` URL.

* **`master.endpoints[].endpoint`**
  Type: `string`. Required.
  HBase master JMX endpoint as either a `host:port` RMI registry address or a `service:jmx:rmi`
  URL.

* **`regionserver`**
  Type: `object`. Optional.
  Settings for collecting from HBase RegionServers. RegionServers are discovered from workload
  annotations, or listed here as endpoints.

* **`regionserver.enabled`**
  Type: `boolean`. Optional. Default: `true`.
  Whether this instance collects from HBase RegionServers. If true, the Collector collects from
  the RegionServers it discovers or the targets in `regionserver.endpoints`. If false, the
  Collector doesn't collect from RegionServers, and master collection isn't affected.

* **`regionserver.endpoints`**
  Type: `array of object`. Optional. Default: `[]`.
  HBase RegionServer Java Management Extensions (JMX) targets. When this list is empty or
  omitted, the integration uses automatic target discovery. When the list contains targets, this
  integration instance collects from exactly those targets and doesn't automatically discover
  RegionServers. Configure a second named instance to use both methods. Each endpoint can be a
  `host:port` Remote Method Invocation (RMI) registry address or a `service:jmx:rmi` URL.

* **`regionserver.endpoints[].endpoint`**
  Type: `string`. Required.
  HBase RegionServer JMX endpoint as either a `host:port` RMI registry address or a
  `service:jmx:rmi` URL.

* **`registry_tls`**
  Type: `boolean`. Optional. Default: `false`.
  Whether to use Transport Layer Security (TLS) for the RMI registry connection. If true, the
  Collector uses TLS. If false, the Collector uses an unencrypted registry connection.

* **`advertised_host_override`**
  Type: `string`. Optional. Default: empty string.
  Host the Collector uses for RMI server connections instead of the host advertised by returned
  RMI stubs. When empty, the Collector uses the registry endpoint host.

* **`advertised_port_override`**
  Type: `integer`. Optional. Default: `0`. Minimum: `0`. Maximum: `65535`.
  Port the Collector uses for RMI server connections instead of the port advertised by returned
  RMI stubs. When set to `0`, the Collector uses the advertised port.

* **`username`**
  Type: `string`. Optional.
  Username for JMX authentication, used for every master and RegionServer. Configure `username`
  and `password` together.

* **`password`**
  Type: `string`. Optional.
  Password for JMX authentication, used for every master and RegionServer. Configure `username`
  and `password` together.

* **`collection_interval`**
  Type: `duration`. Optional. Default: `15s`.
  How often the Collector collects metrics from each HBase JMX endpoint.

* **`bean_detection_interval`**
  Type: `duration`. Optional. Default: `10m`.
  How often the Collector refreshes the set of available JMX beans.

* **`bean_limit`**
  Type: `integer`. Optional. Default: `35000`. Minimum: `1`.
  Maximum number of JMX bean matches collected from each endpoint. When the number of matches
  exceeds this value, the integration drops beans according to the attribute mapping priority.

* **`target_system`**
  Type: `string`. Optional. Default: `hbase`.
  Comma-delimited attribute mapping names to apply. The HBase mapping, which covers both masters
  and RegionServers, is enabled by default, and the Java Virtual Machine (JVM) mapping is
  included automatically. Add `-jvm` to disable JVM metrics when at least one other mapping is
  configured.

* **`attribute_mapping_paths`**
  Type: `array of string`. Optional. Default: `[]`.
  Directories and files containing custom JMX attribute mappings. The Collector searches these
  paths before the embedded HBase and JVM mappings.

* **`tls`**
  Type: `object`. Optional. Default: `{}`.
  TLS certificate and client authentication settings for RMI registry and server connections.

* **`tls.ca_file`**
  Type: `string`. Optional.
  Path to a PEM-encoded certificate authority (CA) certificate file used to verify the JMX
  server certificate. If omitted, the Collector uses the system certificate authority pool.

* **`tls.ca_pem`**
  Type: `string`. Optional.
  PEM-encoded certificate authority certificate content used to verify the JMX server
  certificate.

* **`tls.cert_file`**
  Type: `string`. Optional.
  Path to the PEM-encoded client certificate file to present when the JMX server requires mutual
  TLS authentication.

* **`tls.cert_pem`**
  Type: `string`. Optional.
  PEM-encoded client certificate content to present when the JMX server requires mutual TLS
  authentication.

* **`tls.cipher_suites`**
  Type: `array of string`. Optional.
  TLS cipher suites that the Collector can use, in preference order. If omitted, the Collector
  uses secure default cipher suites. For supported names, see
  [https://go.dev/src/crypto/tls/cipher\_suites.go](https://go.dev/src/crypto/tls/cipher_suites.go).

* **`tls.curve_preferences`**
  Type: `array of string`. Optional.
  Elliptic curves that the Collector can use for an elliptic curve Diffie-Hellman ephemeral
  handshake, in preference order. If omitted, the Collector uses its default curves.

* **`tls.include_system_ca_certs_pool`**
  Type: `boolean`. Optional.
  Whether to add certificates from the system certificate authority pool to the configured
  certificate authorities. If true, the Collector trusts both sources. If false, the Collector
  trusts only the configured certificate authorities.

* **`tls.insecure`**
  Type: `boolean`. Optional.
  Whether to disable transport security. This setting must be false. Use `registry_tls` to
  control transport security for the RMI registry.

* **`tls.insecure_skip_verify`**
  Type: `boolean`. Optional.
  Whether to skip verification of the JMX server certificate. If true, the connection is
  encrypted but isn't authenticated and is vulnerable to man-in-the-middle attacks. If false,
  the Collector verifies the certificate.

* **`tls.key_file`**
  Type: `string`. Optional.
  Path to the PEM-encoded client private key file used with `cert_file` for mutual TLS
  authentication.

* **`tls.key_pem`**
  Type: `string`. Optional.
  PEM-encoded client private key content used with `cert_pem` for mutual TLS authentication.

* **`tls.max_version`**
  Type: `string`. Optional.
  Maximum TLS protocol version the Collector can use. If omitted, the Collector uses the highest
  TLS version it supports.

* **`tls.min_version`**
  Type: `string`. Optional.
  Minimum TLS protocol version the Collector can use. If omitted, the minimum version is TLS
  1.2.

* **`tls.reload_interval`**
  Type: `duration`. Optional.
  How often the Collector reloads certificate and key files. If omitted, the Collector doesn't
  reload the files.

* **`tls.server_name_override`**
  Type: `string`. Optional.
  Server name to verify in the JMX certificate instead of the endpoint hostname. Use this
  setting when the certificate name differs from the endpoint hostname.

* **`tls.tpm`**
  Type: `object`. Optional.
  Trusted platform module settings for the TLS private key.

* **`tls.tpm.auth`**
  Type: `string`. Optional.
  Authorization value for the trusted platform module key.

* **`tls.tpm.enabled`**
  Type: `boolean`. Optional.
  Whether to use a trusted platform module for the TLS private key. If true, the Collector loads
  the key from the configured device or socket. If false, the Collector uses the configured key
  file or in-memory key.

* **`tls.tpm.owner_auth`**
  Type: `string`. Optional.
  Owner authorization value for the trusted platform module.

* **`tls.tpm.path`**
  Type: `string`. Optional.
  Path to the trusted platform module device or Unix domain socket, such as `/dev/tpm0` or
  `/dev/tpmrm0`.


## Related topics

- [CXDOT Collector integrations](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector.md)
- [Apache](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/apache.md)
- [Apache Mesos](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/mesos.md)
- [Apache Tomcat](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/tomcat.md)
- [Apache Cassandra](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/cassandra.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.