> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Confluent Cloud source plugin

> Configure the Confluent Cloud source plugin in Palo Alto Networks Telemetry Pipeline to ingest logs for processing and routing.

export const entity_0 = "Confluent Cloud source plugin"

export const plugin_0 = "Confluent Cloud source plugin"

<Info>
  This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone
  product separate from Palo Alto Networks Cortex XCOR.
</Info>

The Confluent Cloud [source plugin](/ingest/pipeline/plugins/source-plugins)
(name: `kafka`, alias: `confluent_cloud`) lets you retrieve data from Confluent
Cloud and ingest it into a telemetry pipeline.

This is a
[pull-based](/ingest/pipeline/plugins/source-plugins#push-based-and-pull-based-source-plugins)
source plugin.

## Supported telemetry types

The {plugin_0} for Palo Alto Networks Telemetry Pipeline supports these telemetry types:

| Logs | Metrics | Traces |
| :-: | :-: | :-: |
| <Icon icon="circle-check" color="green" /> | <Icon icon="ban" color="red" /> | <Icon icon="ban" color="red" /> |

## Configuration parameters

Use the parameters in this section to configure the {entity_0}. The
Palo Alto Networks Telemetry Pipeline web interface uses the items in the **Name** column to
describe these parameters. [Pipeline configuration files](/ingest/pipeline/v2/configure/config-files)
use the items in the **Key** column as YAML keys.

### General

| Name | Key | Description | Default |
| - | - | - | - |
| **Confluent Cloud Bootstrap Servers** | `brokers` | Required. The Confluent Cloud bootstrap found within the configuration settings. | `[YOUR_BOOTSTRAP_SERVER].confluent.cloud:9092` |
| **Confluent Cloud Topic** | `topics` | Required. The Confluent Cloud topic to read information from. | *none* |
| **Confluent Cloud API Key** | `rdkafka.sasl.username` | Your Confluent Cloud API key. | *none* |
| **Confluent Cloud API Secret** | `rdkafka.sasl.password` | Required. Your Confluent Cloud API secret. | *none* |

### Advanced

| Name | Key | Description | Default |
| - | - | - | - |
| **Minimum Queued Messages** | `rdkafka.queued.min.messages` | The minimum number of messages per topic and partition that Palo Alto Networks Telemetry Pipeline tries to maintain in the local consumer queue. | `10` |
| **Session Timeout (ms)** | `rdkafka.session.timeout.ms` | How long Palo Alto Networks Telemetry Pipeline waits before terminating a session connection. | `45000` |
| **Security Protocol** | `rdkafka.security.protocol` | The security protocol for Azure Event Hub. If you require OAuth 2.0 or OpenID authentication, contact Cortex XCOR Support. | `SASL_SSL` |
| **SASL Mechanism** | `rdkafka.sasl.mechanism` | The transport mechanism for the SASL connection. | `PLAIN` |
| **Memory Buffer Limit** | `mem_buf_limit` | Sets a limit for how much buffered data the plugin can write to memory, which affects backpressure. This value must follow Fluent Bit's rules for [unit sizes](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit#unit-sizes). If unspecified, no limit is enforced. <p />For v2 pipelines, this parameter affects only pipelines with the Deployment or DaemonSet [workload](/ingest/pipeline/v2/configure/kubernetes/workloads) type. To learn more, see the v2 [backpressure](/ingest/pipeline/v2/configure/backpressure) guide. <p />For v3 pipelines, this parameter is independent from the [OpenTelemetry `memory_limiter` and `batch` processors](/ingest/pipeline/v3#route-data-from-fluent-bit-to-opentelemetry). | *none* |

### Security and TLS

| Name | Key | Description | Default |
| - | - | - | - |
| **TLS** | `tls` | If `true`, enables TLS/SSL. If `false`, disables TLS/SSL. Accepted values: `true`, `false`. | `false` |
| **TLS Certificate Validation** | `tls.verify` | If `on`, and if `tls` is `true`, enables TLS/SSL certificate validation. If `off`, disables TLS/SSL certificate validation. Accepted values: `on`, `off`. | `on` |
| **TLS Debug Level** | `tls.debug` | Sets TLS debug verbosity level. Accepted values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational), `4` (Verbose). | `1` |
| **CA Certificate File Path** | `tls.ca_file` | Absolute path to CA certificate file. | *none* |
| **Certificate File Path** | `tls.crt_file` | Absolute path to certificate file. | *none* |
| **Private Key File Path** | `tls.key_file` | Absolute path to private key file. | *none* |
| **Private Key Path Password** | `tls.key_passwd` | Password for private key file. | *none* |
| **TLS SNI Hostname Extension** | `tls.vhost` | Hostname to be used for TLS SNI extension. | *none* |

### Other

This parameter doesn't have an equivalent setting in the Palo Alto Networks Telemetry Pipeline web
interface, but you can use it in pipeline configuration files.

| Name | Key | Description | Default |
| - | - | - | - |
| *none* | `buffer_max_size` | Sets the maximum [chunk](https://docs.fluentbit.io/manual/administration/buffering-and-storage#chunks) size for buffered data. If a single log exceeds this size, the plugin drops that log. | `4M` |

### Extended librdkafka parameters

This plugin uses the [librdkafka](https://github.com/confluentinc/librdkafka)
library. Certain configuration parameters available through the Palo Alto Networks
Telemetry Pipeline UI are based on librdkafka settings. These parameters generally
use the `rdkafka.` prefix.

In addition to the parameters available through the
Palo Alto Networks Telemetry Pipeline UI, you can
customize any of the
[librdkafka configuration properties](https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md)
by adding them to a pipeline configuration file. To do so, append the `rdkafka.`
prefix to the name of that property.

For example, to customize the `socket.keepalive.enable` property, add the
`rdkafka.socket.keepalive.enable` key to your configuration file.

<Note>
  Don't use librdkafka properties to configure a pipeline's memory buffer. Instead,
  use the [`buffer_max_size`](#other) parameter.
</Note>


## Related topics

- [Confluent Cloud DataSet destination plugin](/ingest/pipeline/plugins/destination-plugins/confluent-cloud.md)
- [Ingest telemetry data with source plugins](/ingest/pipeline/plugins/source-plugins.md)
- [Send telemetry data with destination plugins](/ingest/pipeline/plugins/destination-plugins.md)
- [Devo Cloud destination plugin](/ingest/pipeline/plugins/destination-plugins/devo.md)
- [Cloudflare LogPush source plugin](/ingest/pipeline/plugins/source-plugins/cloudflare.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.