Supported telemetry types
The Argo CD integration supports these telemetry types:Prerequisites
The Argo CD integration has the following prerequisites:-
Make the metrics port of each Argo CD component reachable from the CXDOT Collector.
If a NetworkPolicy restricts ingress to the Argo CD pods, allow the CXDOT Collector
to connect to these ports. By default, Argo CD serves metrics on the following ports:
- Application Controller:
8082 - ApplicationSet Controller:
8080 - API Server:
8083 - Repository Server:
8084 - Notifications Controller:
9001 - Commit Server:
8087
- Application Controller:
Configure
The Argo CD integration is enabled by default. To choose the components that it collects from, add a CXDOT integration annotation to the Argo CD pods or their metrics Services in Kubernetes. Alternatively, list the metrics endpoints in your Helm values. To configure the Argo CD integration, follow these steps:-
To collect from Argo CD pods in Kubernetes, add a CXDOT integration annotation to the
pod template of each component. The annotation key must end with the container name,
and each endpoint names the component that serves it. For example, add the following
annotation to your Application Controller StatefulSet:
The Collector replaces
${target.host}with the pod IP address and${target.port.metrics}with the container port namedmetrics. Metrics from each annotated pod include that pod’s Kubernetes metadata. With the Argo CD Helm chart, add the annotation under thepodAnnotationsvalue of each component:controller,applicationSet,server,repoServer,notifications, andcommitServer. The chart names each component’s container after itscomponentvalue, such asrepo-server, and its metrics portmetrics, so the annotation for each component differs from the example only in the container name and thecomponentvalue. -
Optional: If your Argo CD containers don’t name their metrics ports, as most
containers in the published installation manifests don’t, configure the metrics
Service of each component with a CXDOT integration annotation instead. The
integrations.cxdot.paloaltonetworks.com/endpointskey makes each ready pod behind the Service a separate target. For example, add the following annotation to theargocd-metricsService, which serves the Application Controller’s metrics:The Collector replaces${target.host}with the IP address of each pod and${target.port.metrics}with the pod port that the Service port namedmetricsforwards to. In the published manifests, the metrics Services of the other components areargocd-applicationset-controller,argocd-server-metrics,argocd-repo-server,argocd-notifications-controller-metrics, andargocd-commit-server, and each one names its metrics portmetrics. -
Optional: To collect from Argo CD components in another Kubernetes cluster or at
addresses that you choose, configure static targets. List the metrics endpoint of
each component in
endpoints, with the component that serves it. For example, add the following to the Helm values file for your CXDOT Collector:When you list endpoints, the integration collects from exactly those endpoints instead of from annotated pods and Services. In Kubernetes, the elected cluster Collector collects from each endpoint that you list, one time rather than once per node. For more information, see architecture. Metrics from listed endpoints don’t carry Kubernetes metadata from the Argo CD pods. Use theargocd.componentandurl.fullattributes to identify each endpoint. -
Optional: If your Argo CD metrics endpoints use HTTPS or require a bearer token,
configure the integration to match. Set
bearer_token_fileto the path of a file that holds the token, and usetlsto verify the server certificate. These settings requireschema_version: v1.18or later, in your Helm values or in the CXDOT integration annotation. For example, add the following to the Helm values file for your CXDOT Collector:The Collector sends the token in theAuthorizationheader of every request to the Argo CD components and rereads the file when it changes, so you can rotate the token without restarting the Collector. Thetlssettings apply to HTTPS endpoints only. Settls.ca_fileto a single PEM file that holds the certificate authority (CA) certificates. The Collector doesn’t read a directory of certificates. Mount the token and the CA certificate file into the Collector pods that collect from the endpoints, for example from a Kubernetes Secret, with theextraVolumesandextraVolumeMountsvalues of theclusterCollectorornodeCollector. The Collector doesn’t support basic authentication with a username and password for this integration. For annotated pods and Services, set these settings in the CXDOT integration annotation instead. The Collector rejects an annotation that setstls.ca_file,bearer_token_file, or any other setting that names a file, unless the file is undercollection.cxdot.integrations.discovery.kubernetes.annotation_restrictions.allowed_collector_file_pathsin your Helm values, which allows no paths by default. The node Collector collects from annotated pods and the cluster Collector from annotated Services, so mount the file into the Collector for each kind of annotation that names it. For example, the following Helm values mount a Kubernetes Secret into both Collectors and allow its path:Then setbearer_token_file: /etc/cxdot/argocd/tokenin theargocdblock of the annotation, wheretokenis the key that holds the token in the Kubernetes Secret, and set the annotation’sschema_versiontov1.18or later. The Secret must be in the namespace of your CXDOT Collector. After you allow a path, an annotation on any pod or Service in your cluster can name the file, and the Collector sends the token to that annotation’s target. Thecustom_pod_host: denyandcustom_service_host: denysettings limit each annotation’s target to the annotated pod’s or Service’s own addresses, so the token reaches only annotated workloads. Allow only paths that hold secrets that you intend annotated workloads to use. -
Optional: To collect from both annotated pods and listed endpoints, list the
endpoints in a second integration instance. For example, add the following to the
Helm values file for your CXDOT Collector:
The
argocd/remotekey defines a named instance for the listed endpoints. The bareargocdkey lists no endpoints, so the integration continues to collect from annotated pods and Services. -
Optional: If the OpenMetrics integration also scrapes your Argo CD pods or Services,
disable OpenMetrics on that workload to avoid duplicate metrics. For example, set
openmetrics: {enabled: false}in a CXDOT integration annotation on the pod or Service. For more information, see autodiscovery.
Validate
To validate the Argo CD integration, follow these steps:-
In the Live Telemetry Analyzer,
filter for
__name__=cxdot.integration.target.healthandcxdot.integration.name=argocd. Confirm that each component reports1forcxdot.integration.target.health, identified by itsargocd.componentandurl.fullattributes. -
In Metrics Explorer,
run the following query:
Confirm that the query returns a time series for each combination of sync status and health status that your Applications report.
Troubleshooting
- A component reports
0forcxdot.integration.target.health: Confirm that the component meets the requirements in the Prerequisites section and that its endpoint uses the component’s metrics port, such as8083for the API Server rather than8080, the port that serves its API. argocd.app.labels,argocd.app.condition, orargocd.appset.labelsdoesn’t appear: Argo CD reports these metrics only for the labels and conditions that you name. Set--metrics-application-labelsand--metrics-application-conditionson the Application Controller, and--metrics-applicationset-labelson the ApplicationSet Controller. For more information, see Metrics in the Argo CD documentation.- GitHub API metrics don’t appear: The ApplicationSet Controller reports
argocd.github_api.*metrics only for generators that call the GitHub API, and only after you setapplicationsetcontroller.enable.github.api.metrics: "true"in theargocd-cmd-params-cmConfigMap. - Commit Server metrics don’t appear: Argo CD runs the Commit Server only when the source hydrator is enabled.
Configuration reference
Configure one Argo CD integration instance with the following settings. In Helm values, place these settings undercollection.cxdot.integrations.argocd. In a Collector configuration file,
place them under cxdot.integrations.argocd.
Optional settings
-
enabledType:boolean. Optional. Default:true. Whether to enable this Argo CD integration instance. If true, the Collector collects metrics from the Argo CD components that this instance is configured for. If false, the Collector doesn’t run this integration instance. -
endpointsType:array of object. Optional. Default:[]. Argo CD components to monitor, by the address of each component’s Prometheus metrics endpoint. Each entry is one target, so list an address that reaches one component pod. An address that balances requests across replicas, such as a Kubernetes Service with several pods, reaches a different replica on each collection. -
endpoints[].endpointType:string. Required. Address of the component’s metrics endpoint. Specifyhost:portto request the/metricspath over HTTP, or a full HTTP or HTTPS URL. The Collector requests a URL with a path as given, and requests/metricsfor a URL without a path. By default, Argo CD serves metrics on port8082for the Application Controller,8080for the ApplicationSet Controller,8083for the API Server,8084for the Repository Server,9001for the Notifications Controller, and8087for the Commit Server. -
endpoints[].componentType:string. Required. The Argo CD component that serves the endpoint. The Collector reports it as theargocd.componentattribute on every metric from the endpoint. The values are the component names that Argo CD’s manifests use in theapp.kubernetes.io/componentlabel, whereserveris the API Server. Allowed values:application-controller,applicationset-controller,server,repo-server,notifications-controller,commit-server. -
collection_intervalType:duration. Optional. Default:15s. How often the integration collects metrics from each Argo CD component. -
timeoutType:duration. Optional. Default:10s. Maximum time the Collector waits for one collection from a component. The value must be greater than zero. A value longer thancollection_intervalis lowered tocollection_interval. -
max_samplesType:integer. Optional. Default:0. Minimum:0. Maximum number of samples accepted from one metric collection. A value of0, the default, means no limit. A collection that exceeds the limit fails and reports the target as unhealthy instead of returning an incomplete set of samples. The Application Controller reports several time series for each Application, so its sample count grows with the number of Applications. -
follow_redirectsType:boolean. Optional. Default:true. Whether to follow HTTP redirects. If true, the Collector follows redirects. If false, a redirect causes the metric collection to fail. -
bearer_token_fileType:string. Optional. Default: empty string. Path to a file that holds a bearer token. The Collector sends the token in theAuthorizationheader of every request to the Argo CD components and rereads the file when it changes. If the metrics endpoints don’t require a token, omit this setting. -
tlsType:object. Optional. Transport layer security (TLS) settings for HTTPS endpoints. The Collector verifies certificates unlessinsecure_skip_verifyistrue. These settings don’t apply to HTTP endpoints. -
tls.ca_fileType:string. Optional. Path to a single PEM file that holds the certificate authority (CA) certificates used to verify the Argo CD component’s server certificate. A directory of certificates isn’t supported. If set, the Collector trusts only the certificate authorities in this file instead of the system certificate authority pool. -
tls.insecure_skip_verifyType:boolean. Optional. Whether to skip verification of the Argo CD component’s server certificate. If true, the connection is encrypted but isn’t authenticated and is vulnerable to man-in-the-middle attacks. If false, the Collector verifies the server certificate. -
tls.server_nameType:string. Optional. Server name to verify in the Argo CD component’s server certificate instead of the endpoint hostname. The Collector also sends this name as the TLS server name indication. Use this setting when the certificate name differs from the endpoint hostname.