Amazon Kinesis Firehose source plugin - Cortex XCOR Documentation

This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone product separate from Palo Alto Networks Cortex XCOR.

The Amazon Kinesis Firehose source plugin (name: http, alias: AWS_Kinesis_Firehose) lets you retrieve data from your Amazon Kinesis Firehose instances and ingest it into a telemetry pipeline. This is a push-based source plugin.

Supported telemetry types

The Amazon Kinesis Firehose source plugin for Palo Alto Networks Telemetry Pipeline supports these telemetry types:

Logs	Metrics	Traces

Configuration parameters

Use the parameters in this section to configure the Amazon Kinesis Firehose source plugin. The Palo Alto Networks Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.

General

Name	Key	Description	Default
Port	`port`	Required. The port to listen to for incoming Kinesis Firehose messages.	`9880`

Advanced

Name	Key	Description	Default
Tag Key	`tag_key`	Specifies the key name to overwrite a tag. If set, the value of that key overwrites the tag.	none
Buffer Max Size	`buffer_max_size`	Sets the maximum chunk size for buffered data. If a single record exceeds this size, the plugin drops that record.	`4M`
Buffer Chunk Size	`buffer_chunk_size`	Sets the default chunk size for buffered data. If a single record exceeds this size, the plugin temporarily increases the chunk size up to the value of `buffer_max_size` to accommodate it.	`512k`
Successful Response Code	`successful_response_code`	Sets a successful response code. Accepted values: `200`, `201`, `204`.	`200`
Memory Buffer Limit	`mem_buf_limit`	Sets a limit for how much buffered data the plugin can write to memory, which affects backpressure. This value must follow Fluent Bit’s rules for unit sizes. If unspecified, no limit is enforced. For v2 pipelines, this parameter affects only pipelines with the Deployment or DaemonSet workload type. To learn more, see the v2 backpressure guide. For v3 pipelines, this parameter is independent from the OpenTelemetry `memory_limiter` and `batch` processors.	none

Security and TLS

Name	Key	Description	Default
TLS	`tls`	If `true`, enables TLS/SSL. If `false`, disables TLS/SSL. Accepted values: `true`, `false`.	`false`
TLS Certificate Validation	`tls.verify`	If `on`, and if `tls` is `true`, enables TLS/SSL certificate validation. If `off`, disables TLS/SSL certificate validation. Accepted values: `on`, `off`.	`on`
TLS Debug Level	`tls.debug`	Sets TLS debug verbosity level. Accepted values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), `4` (Verbose).	`1`
CA Certificate File Path	`tls.ca_file`	Absolute path to CA certificate file.	none
Certificate File Path	`tls.crt_file`	Absolute path to certificate file.	none
Private Key File Path	`tls.key_file`	Absolute path to private key file.	none
Private Key Path Password	`tls.key_passwd`	Password for private key file.	none
TLS SNI Hostname Extension	`tls.vhost`	Hostname to be used for TLS SNI extension.	none