Google Cloud BigQuery destination plugin - Cortex XCOR Documentation

This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone product separate from Palo Alto Networks Cortex XCOR.

The Google Cloud BigQuery destination plugin (name: bigquery) lets you configure your telemetry pipeline to send your log data and metrics to Google Cloud BigQuery.

Supported telemetry types

The Google Cloud BigQuery destination plugin for Palo Alto Networks Telemetry Pipeline supports these telemetry types:

Logs	Metrics	Traces

Configuration parameters

Use the parameters in this section to configure the Google Cloud BigQuery destination plugin. The Palo Alto Networks Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.

General

Name	Key	Description	Default
Google Service Credentials Path	`google_service_credentials`	Required. The Service Credentials file lets Palo Alto Networks Telemetry Pipeline communicate directly with Google Cloud Services. For information about how to set up service credentials, see the Google Cloud Observability documentation.	none
Google Project Id	`project_id`	The project ID containing the BigQuery dataset to stream into. If the service file is provided then the project ID is taken from there.	none
Existing Data Set ID	`dataset_id`	The dataset ID of the BigQuery dataset to write into. This dataset must exist in your project.	none
Existing Table ID	`table_id`	The table ID of the BigQuery table to write into. This table must exist in the specified dataset and the schema must match the output.	none

Advanced

Name	Key	Description	Default
Skip Invalid Rows	`skip_invalid_rows`	If `true`, inserts all valid rows of a request, even if invalid rows exist. If `false`, causes the entire request to fail if any invalid rows exist. Accepted values: `true`, `false`.	`false`
Ignore Unknown Values	`ignore_unknown_values`	Accepts rows that contain values that don’t match the schema, and ignores unknown values. When set to `false`, treats unknown values as errors. Accepted values: `true`, `false`.	`false`
Enable Workload Identity Federation	`enable_workload_identity_federation`	Enables workload identity federation as an alternative authentication method. Can’t be used with service account credentials file or environment variable. AWS is the only supported identity provider. Accepted values: `true`, `false`.	`false`
AWS Region for BigQuery	`aws_region`	Google Cloud region for BigQuery.	none
GCP Project Number	`project_number`	Google Cloud project number where the identity provider was created. Used to construct the full resource name of the identity provider.	none
GCP Pool Id	`pool_id`	Google Cloud workload identity pool where the identity provider was created. Used to construct the full resource name of the identity provider.	none
GCP Provider Id	`provider_id`	Google Cloud workload identity provider. Used to construct the full resource name of the identity provider. Only AWS accounts are supported.	none

Security and TLS

Name	Key	Description	Default
TLS	`tls`	If `true`, enables TLS/SSL. If `false`, disables TLS/SSL. Accepted values: `true`, `false`.	`false`
TLS Certificate Validation	`tls.verify`	If `on`, and if `tls` is `true`, enables TLS/SSL certificate validation. If `off`, disables TLS/SSL certificate validation. Accepted values: `on`, `off`.	`on`
TLS Debug Level	`tls.debug`	Sets TLS debug verbosity level. Accepted values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), `4` (Verbose).	`1`
CA Certificate File Path	`tls.ca_file`	Absolute path to CA certificate file.	none
Certificate File Path	`tls.crt_file`	Absolute path to certificate file.	none
Private Key File Path	`tls.key_file`	Absolute path to private key file.	none
Private Key Path Password	`tls.key_passwd`	Password for private key file.	none
TLS SNI Hostname Extension	`tls.vhost`	Hostname to be used for TLS SNI extension.	none

Advanced Networking

Name	Key	Description	Default
DNS Mode	`net.dns.mode`	Selects the primary DNS connection type, which can be `TCP` or `UDP`.	none
DNS Resolver	`net.dns.resolver`	Selects the primary DNS connection type, which can be `LEGACY` or `ASYNC`.	none
Prefer IPv4	`net.dns.prefer_ipv4`	Prioritizes IPv4 DNS results when trying to establish a connection. Accepted values: `true`, `false`.	`false`
Keepalive	`net.keepalive`	Enables or disables Keepalive support. Accepted values: `true`, `false`.	`true`
Keepalive Idle Timeout	`net.keepalive_idle_timeout`	Sets the maximum time allowed for an idle Keepalive connection.	`30s`
Max Connect Timeout	`net.connect_timeout`	Sets the maximum time allowed to establish a connection, which includes the TLS handshake.	`10s`
Max Connect Timeout Log Error	`net.connect_timeout_log_error`	Specifies whether to log an error on connection timeout. When disabled, the timeout is logged as a debug message. Accepted values: `true`, `false`.	`true`
Source Address	`net.source_address`	Specifies the network address to bind for data traffic.	none
Max Keepalive Recycle	`net.keepalive_max_recycle`	Sets the maximum number of times a keepalive connection can be used before it’s retired.	`2000`