Concourse CI

The Concourse CI integration requires CXDOT Collector 1.13.0 or greater. Concourse is an open source continuous integration system that runs pipelines of resources and jobs on a pool of workers. Use the Concourse CI integration to collect build, job scheduling, resource check, worker, garbage collection, database, and API metrics from the Concourse web nodes running in your environment. The Concourse CI integration supports Concourse 7.11 or greater.

​

Supported telemetry types

The Concourse CI integration supports these telemetry types:

​

Choose a collection method

The CXDOT Collector collects Concourse metrics in one of two ways:

• Recommended: Turn on the Prometheus emitter on each web node and scrape its endpoint with the Concourse CI integration. You get the Concourse CI integration’s metrics, its overview dashboard, and a health metric for each web node. The Prerequisites and Configure sections cover this method.
• Alternative: Keep the Datadog emitter and send its DogStatsD metrics to the Datadog integration. Choose this method if you can’t switch a web node to the Prometheus emitter yet. You get the metric names that the emitter sends, and none of the Concourse CI integration’s metrics, dashboard, or health metric. The Send DogStatsD metrics to the Datadog integration section covers this method.

​

Prerequisites

The Concourse CI integration has the following prerequisites:

• Turn on the Prometheus metrics emitter on each Concourse web node by setting CONCOURSE_PROMETHEUS_BIND_IP and CONCOURSE_PROMETHEUS_BIND_PORT, or the --prometheus-bind-ip and --prometheus-bind-port flags of concourse web. With the Concourse Helm chart, set concourse.web.prometheus.enabled to true, which binds port 9391 and creates a Service for it. For more information, see Metrics in the Concourse documentation. A web node runs one metrics emitter, and a web node with more than one emitter configured fails to start with the error multiple emitters configured. To turn on the Prometheus emitter on a web node that sends its metrics through another emitter, remove that emitter’s settings. The Prometheus emitter serves an endpoint that any number of readers can request, so move whatever reads the node’s metrics today to the Prometheus endpoint when you switch. To confirm that a web node serves metrics, request /metrics on the Prometheus port from a machine that can reach it and confirm that the request returns a successful status code:
  curl -i http://concourse-web.example.com:9391/metrics
  
• Make the Prometheus port reachable on each web node from the CXDOT Collector.

​

Configure

The Concourse CI integration is enabled by default, but it collects nothing until you list the web nodes to monitor. The integration doesn’t discover Concourse pods. In Kubernetes, the elected cluster Collector collects from each endpoint that you list, one time rather than once per node. For more information, see architecture. To configure the Concourse CI integration, follow these steps:

1. List each Concourse web node in endpoints. For example, add the following to the Helm values file for your CXDOT Collector:
   config:
     integrations:
       concourse_ci:
         endpoints:
           - endpoint: concourse-web-1.example.com:9391
           - endpoint: concourse-web-2.example.com:9391
           - endpoint: http://concourse-web-3.example.com:9391/metrics
   
   List an address that reaches one web node. Each request to an address that balances across replicas, such as a Kubernetes Service in front of several web pods, reaches a different replica, so the metrics mix the replicas’ values. With one web node, the Service that the Concourse Helm chart creates for the Prometheus port reaches that node. For a Concourse deployment with several web nodes, list each node’s own address. Metrics from listed endpoints don’t carry Kubernetes metadata from the Concourse pods. Use the server.address and server.port attributes to identify each web node.
2. Optional: If an address you list reaches a Concourse web pod, or a Service in front of one, that carries autodiscovery annotations for OpenMetrics collection, remove those annotations. Otherwise, the OpenMetrics integration also collects the pod’s metrics, and the metrics will be duplicated. For more information, see autodiscovery.

​

Validate

To validate the Concourse CI integration, follow these steps:

1. In the Live Telemetry Analyzer, filter for __name__=cxdot.integration.target.health and cxdot.integration.name=concourse_ci. Confirm that each endpoint that you listed reports 1, identified by its server.address and server.port attributes.
2. In Metrics Explorer, run the following query after Concourse finishes builds:
   sum by ("server.address", "concourse.build.status") (rate({"concourse.build.finished"}[$__rate_interval]))
   
   Confirm that the query returns a time series for each web node and build status.

​

Send DogStatsD metrics to the Datadog integration

Set CONCOURSE_DATADOG_AGENT_HOST and CONCOURSE_DATADOG_AGENT_PORT, or the --datadog-agent-host and --datadog-agent-port flags of concourse web, to the DogStatsD listener of the Datadog integration. The listener accepts UDP on port 9125 by default. The Concourse CI integration doesn’t collect these DogStatsD metrics. The emitter sends each metric as a gauge named with the --datadog-prefix value followed by the event name, such as concourse.ci.builds_running with the Concourse Helm chart’s default prefix. The metric names on this page and its dashboard don’t apply to those metrics.

​

Configuration reference

Configure one Concourse CI integration instance with the following settings. In Helm values, place these settings under config.integrations.concourse_ci. In a Collector configuration file, place them under cxdot.integrations.concourse_ci.

​

Optional settings

• enabled Type: boolean. Optional. Default: true. Whether to enable this Concourse CI integration instance. If true, the Collector collects metrics from the web nodes that you list. If false, the Collector doesn’t run this integration instance.
• endpoints Type: array of object. Optional. Default: []. Concourse web nodes to monitor, by the address of each node’s Prometheus metrics endpoint. The integration collects only from the endpoints that you list and doesn’t discover Concourse pods. It collects nothing until you list at least one endpoint. Specify host:port to request the /metrics path over HTTP, or a full HTTP or HTTPS URL. For a URL without a path, the Collector requests /metrics. A web node serves the endpoint only when its Prometheus emitter is on, set with CONCOURSE_PROMETHEUS_BIND_IP and CONCOURSE_PROMETHEUS_BIND_PORT. The Concourse Helm chart uses port 9391. Each entry is one target, so list an address that reaches one web node. An address that balances requests across replicas, such as a Kubernetes Service with several pods, reaches a different replica on each collection.
• endpoints[].endpoint Type: string. Required. Address of one Concourse web node’s Prometheus metrics endpoint, in host:port format or as a full HTTP or HTTPS URL. For host:port, the Collector requests the /metrics path over HTTP. A full URL is requested as given, such as http://concourse-web.example.com:9391/metrics. For a URL without a path, the Collector requests /metrics.
• collection_interval Type: duration. Optional. Default: 15s. How often the integration collects metrics from each Concourse web node.
• timeout Type: duration. Optional. Default: 10s. Maximum time the Collector waits for one collection from a web node. The value must be greater than zero. A value longer than collection_interval is lowered to collection_interval.