> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Concourse CI

> Concourse build, job scheduling, resource check, worker, garbage collection, database and API metrics, scraped from web nodes' Prometheus endpoint.

The Concourse CI integration requires CXDOT Collector 1.13.0 or greater.

[Concourse](https://concourse-ci.org/) is an open source continuous integration system
that runs pipelines of resources and jobs on a pool of workers. Use the Concourse CI
integration to collect build, job scheduling, resource check, worker, garbage
collection, database, and API metrics from the Concourse web nodes running in your
environment.

The Concourse CI integration supports Concourse 7.11 or greater.

## Supported telemetry types

The Concourse CI integration supports these telemetry types:

| Type | Supported |
| - | - |
| Logs | No |
| Metrics | Yes |
| Traces | No |
| Events | No |

## Choose a collection method

The CXDOT Collector collects Concourse metrics in one of two ways:

* Recommended: Turn on the Prometheus emitter on each web node and scrape its endpoint
  with the Concourse CI integration. You get the Concourse CI integration's metrics,
  its overview dashboard, and a health metric for each web node. The Prerequisites and
  Configure sections cover this method.
* Alternative: Keep the Datadog emitter and send its DogStatsD metrics to the Datadog
  integration. Choose this method if you can't switch a web node to the Prometheus
  emitter yet. You get the metric names that the emitter sends, and none of the
  Concourse CI integration's metrics, dashboard, or health metric. The Send DogStatsD
  metrics to the Datadog integration section covers this method.

## Prerequisites

The Concourse CI integration has the following prerequisites:

* Turn on the Prometheus metrics emitter on each Concourse web node by setting
  `CONCOURSE_PROMETHEUS_BIND_IP` and `CONCOURSE_PROMETHEUS_BIND_PORT`, or the
  `--prometheus-bind-ip` and `--prometheus-bind-port` flags of `concourse web`. With
  the Concourse Helm chart, set `concourse.web.prometheus.enabled` to `true`, which
  binds port `9391` and creates a Service for it. For more information, see
  [Metrics](https://concourse-ci.org/docs/operation/metrics/) in the Concourse
  documentation.

  A web node runs one metrics emitter, and a web node with more than one emitter
  configured fails to start with the error `multiple emitters configured`. To turn
  on the Prometheus emitter on a web node that sends its metrics through another
  emitter, remove that emitter's settings. The Prometheus emitter serves an endpoint
  that any number of readers can request, so move whatever reads the node's metrics
  today to the Prometheus endpoint when you switch.

  To confirm that a web node serves metrics, request `/metrics` on the Prometheus
  port from a machine that can reach it and confirm that the request returns a
  successful status code:

  ```shell theme={null}
  curl -i http://concourse-web.example.com:9391/metrics
  ```

* Make the Prometheus port reachable on each web node from the CXDOT Collector.

## Configure

The Concourse CI integration is enabled by default, but it collects nothing until you
list the web nodes to monitor. The integration doesn't discover Concourse pods. In
Kubernetes, the elected cluster Collector collects from each endpoint that you list,
one time rather than once per node. For more information, see
[architecture](https://docs-xcor.paloaltonetworks.com/ingest/xcor/collector/install/kubernetes/architecture.md).

To configure the Concourse CI integration, follow these steps:

1. List each Concourse web node in `endpoints`. For example, add the following to the
   Helm values file for your CXDOT Collector:

   ```yaml theme={null}
   config:
     integrations:
       concourse_ci:
         endpoints:
           - endpoint: concourse-web-1.example.com:9391
           - endpoint: concourse-web-2.example.com:9391
           - endpoint: http://concourse-web-3.example.com:9391/metrics
   ```

   List an address that reaches one web node. Each request to an address that
   balances across replicas, such as a Kubernetes Service in front of several web
   pods, reaches a different replica, so the metrics mix the replicas' values. With
   one web node, the Service that the Concourse Helm chart creates for the Prometheus
   port reaches that node. For a Concourse deployment with several web nodes, list
   each node's own address. Metrics from listed endpoints don't carry Kubernetes
   metadata from the Concourse pods. Use the `server.address` and `server.port`
   attributes to identify each web node.

2. Optional: If an address you list reaches a Concourse web pod, or a Service in
   front of one, that carries autodiscovery annotations for OpenMetrics collection,
   remove those annotations. Otherwise, the OpenMetrics integration also collects
   the pod's metrics, and the metrics will be duplicated. For more information, see
   [autodiscovery](https://docs-xcor.paloaltonetworks.com/ingest/xcor/collector/discover/kubernetes.md).

### Validate

To validate the Concourse CI integration, follow these steps:

1. In the [Live Telemetry Analyzer](https://docs-xcor.paloaltonetworks.com/investigate/analyze/telemetry-analyzer.md),
   filter for `__name__=cxdot.integration.target.health` and
   `cxdot.integration.name=concourse_ci`. Confirm that each endpoint that you listed
   reports `1`, identified by its `server.address` and `server.port` attributes.

2. In [Metrics Explorer](https://docs-xcor.paloaltonetworks.com/investigate/querying/metrics/explorer.md),
   run the following query after Concourse finishes builds:

   ```text theme={null}
   sum by ("server.address", "concourse.build.status") (rate({"concourse.build.finished"}[$__rate_interval]))
   ```

   Confirm that the query returns a time series for each web node and build status.

## Send DogStatsD metrics to the Datadog integration

Set `CONCOURSE_DATADOG_AGENT_HOST` and `CONCOURSE_DATADOG_AGENT_PORT`, or the
`--datadog-agent-host` and `--datadog-agent-port` flags of `concourse web`, to the
DogStatsD listener of the
[Datadog](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/datadog.md)
integration. The listener accepts UDP on port `9125` by default.

The Concourse CI integration doesn't collect these DogStatsD metrics. The emitter sends
each metric as a gauge named with the `--datadog-prefix` value followed by the event
name, such as `concourse.ci.builds_running` with the Concourse Helm chart's default
prefix. The metric names on this page and its dashboard don't apply to those metrics.

## Configuration reference

Configure one Concourse CI integration instance with the following settings. In Helm values,
place these settings under `config.integrations.concourse_ci`. In a Collector configuration
file, place them under `cxdot.integrations.concourse_ci`.

### Optional settings

* **`enabled`**
  Type: `boolean`. Optional. Default: `true`.
  Whether to enable this Concourse CI integration instance. If true, the Collector collects
  metrics from the web nodes that you list. If false, the Collector doesn't run this integration
  instance.

* **`endpoints`**
  Type: `array of object`. Optional. Default: `[]`.
  Concourse web nodes to monitor, by the address of each node's Prometheus metrics endpoint. The
  integration collects only from the endpoints that you list and doesn't discover Concourse
  pods. It collects nothing until you list at least one endpoint. Specify `host:port` to request
  the `/metrics` path over HTTP, or a full HTTP or HTTPS URL. For a URL without a path, the
  Collector requests `/metrics`. A web node serves the endpoint only when its Prometheus emitter
  is on, set with `CONCOURSE_PROMETHEUS_BIND_IP` and `CONCOURSE_PROMETHEUS_BIND_PORT`. The
  Concourse Helm chart uses port `9391`. Each entry is one target, so list an address that
  reaches one web node. An address that balances requests across replicas, such as a Kubernetes
  Service with several pods, reaches a different replica on each collection.

* **`endpoints[].endpoint`**
  Type: `string`. Required.
  Address of one Concourse web node's Prometheus metrics endpoint, in `host:port` format or as a
  full HTTP or HTTPS URL. For `host:port`, the Collector requests the `/metrics` path over HTTP.
  A full URL is requested as given, such as `http://concourse-web.example.com:9391/metrics`. For
  a URL without a path, the Collector requests `/metrics`.

* **`collection_interval`**
  Type: `duration`. Optional. Default: `15s`.
  How often the integration collects metrics from each Concourse web node.

* **`timeout`**
  Type: `duration`. Optional. Default: `10s`.
  Maximum time the Collector waits for one collection from a web node. The value must be greater
  than zero. A value longer than `collection_interval` is lowered to `collection_interval`.


## Related topics

- [CXDOT Collector integrations](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector.md)
- [Service accounts](https://docs-xcor.paloaltonetworks.com/administer/accounts-teams/service-accounts.md)
- [Palo Alto Networks Telemetry Pipeline release notes](https://docs-xcor.paloaltonetworks.com/ingest/pipeline/v2/release-notes.md)
- [Convert Datadog monitors](https://docs-xcor.paloaltonetworks.com/ingest/metrics-traces/collector/mappings/datadog/convert-monitors.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.