Allow records - Cortex XCOR Documentation

This information is for Palo Alto Networks Telemetry Pipeline, which is a standalone product separate from Palo Alto Networks Cortex XCOR.

The allow records processing rule preserves any records that contain a key whose value matches a specified regular expression, and then removes all other records.

Configuration parameters

Use the parameters in this section to configure the allow records processing rule. The Palo Alto Networks Telemetry Pipeline web interface uses the items in the Name column to describe these parameters. Pipeline configuration files use the items in the Key column as YAML keys.

Name	Key	Description	Default
Key	`key`	Required. The key whose value to validate against. If this key isn’t present in a given record, that record is removed.	`log`
Regex	`regex`	Required. The regular expression that determines which records to preserve. This expression is evaluated against the value of the specified key.	none
Match case checkbox	`matchCase`	Indicates whether the regular expression is case-sensitive.	Not selected
Regex engine	`regexEngine`	Required. The engine to parse your regular expression. Accepted values: `GNU`, `Oniguruma`, `PCRE2`, `POSIX`, `TRE`.	`PCRE2`
Comment	`comment`	A custom note or description of the rule’s function. This text is displayed next to the rule’s name in the Actions list in the processing rules interface.	none

Example

Using the allow records rule lets you pare down telemetry data by retaining only the records with key-value pairs that you explicitly choose to keep. For example, given this sample log data:

{"timestamp":"2023-03-28T09:08:41.64283645Z","user_id":3,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:42.643343109Z","user_id":4,"page_id":10,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:48.643600498Z","user_id":1,"page_id":50,"action":"click"}
{"timestamp":"2023-03-28T09:08:50.643773688Z","user_id":5,"page_id":40,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:51.643932272Z","user_id":1,"page_id":30,"action":"purchase"}
{"timestamp":"2023-03-28T09:08:56.644080944Z","user_id":2,"page_id":40,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.64425954Z","user_id":3,"page_id":30,"action":"click"}
{"timestamp":"2023-03-28T09:09:03.644317046Z","user_id":1,"page_id":20,"action":"view"}
{"timestamp":"2023-03-28T09:09:10.64447719Z","user_id":2,"page_id":50,"action":"purchase"}
{"timestamp":"2023-03-28T09:09:17.644810963Z","user_id":2,"page_id":10,"action":"view"}
{"timestamp":"2023-03-28T09:09:20.644994805Z","user_id":1,"page_id":50,"action":"view"}

A processing rule with the Key value action and the Regex value purchase returns the following result:

{"user_id":3,"page_id":30,"action":"purchase","timestamp":"2023-03-28T09:08:41.64283645Z"}
{"user_id":4,"page_id":10,"action":"purchase","timestamp":"2023-03-28T09:08:42.643343109Z"}
{"user_id":5,"page_id":40,"action":"purchase","timestamp":"2023-03-28T09:08:50.643773688Z"}
{"user_id":1,"page_id":30,"action":"purchase","timestamp":"2023-03-28T09:08:51.643932272Z"}
{"user_id":2,"page_id":50,"action":"purchase","timestamp":"2023-03-28T09:09:10.64447719Z"}

This rule retained the records associated with user purchases and removed the records associated with other actions, such as pageviews or clicks. For a processing rule with the opposite effect, see block records.

​Configuration parameters

​Example

​Related rules

Configuration parameters

Example

Related rules