> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Analyze live traffic traces

> Use Trace Analyzer in Palo Alto Networks Cortex XCOR to view incoming traces.

export const MicroscopeIcon = props => <svg viewBox="0 0 24 24" {...props} className="inline-block h-5 w-5" aria-label="Microscope" style={{
  fill: "#28a561"
}}>
    <path d="M9.46,6.28L11.05,9C8.47,9.26 6.5,11.41 6.5,14A5,5 0 0,0 11.5,19C13.55,19 15.31,17.77 16.08,16H13.5V14H21.5V16H19.25C18.84,17.57 17.97,18.96 16.79,20H19.5V22H3.5V20H6.21C4.55,18.53 3.5,16.39 3.5,14C3.5,10.37 5.96,7.2 9.46,6.28M12.74,2.07L13.5,3.37L14.36,2.87L17.86,8.93L14.39,10.93L10.89,4.87L11.76,4.37L11,3.07L12.74,2.07Z" />
  </svg>;

Trace Analyzer provides a real-time view of incoming spans as Palo Alto Networks
Cortex XCOR processes them. Use Trace Analyzer to compare spans by tag, troubleshoot
spikes in ingest rates, and inspect spans before they reach storage.

Trace Explorer queries persisted traces, whereas Trace Analyzer profiles spans in the
trace ingester. Trace Analyzer classifies spans as ingested, dropped, or part of a
trace that's missing its root span. Dropped spans include spans discarded by
[tail sampling](/control/shaping/sample-traces/tail-sampling),
[behavior-based sampling](/control/shaping/sample-traces/behaviors), or product
limits.

## Prerequisites

To explore tracing data in Cortex XCOR, you must either install and
configure the
[Chronosphere Collector](/ingest/metrics-traces/collector/install/tracing) or the
[OpenTelemetry Collector](/ingest/metrics-traces/otel/otel-ingest) to receive trace
data from your services.

## Profile incoming traces

To profile incoming traces:

1. In the navigation menu, click **<Icon icon="shield-user" /> Go to Admin**
   and then select
   **<MicroscopeIcon /> Analyzers <span aria-label="and then">></span> Live Telemetry**.
2. Click the **Traces** tab.
3. Click **Capture live data** to start profiling traces.

   Trace Analyzer automatically pauses capture after three minutes. Click
   **Capture live data** to resume capture.
4. To group incoming spans by tags, select up to 10 tag keys in the
   **Group by tags** field.
5. To filter spans by tag, enter a tag key and value as a `key:value` pair in the
   **Filter by tags** field.

   To display only root spans, enter `__root_service__:true` or
   `__root_operation__:true`. A root span has no parent and represents the top-level
   operation in a trace.

Use [glob syntax](/investigate/querying/glob-syntax) in the **Filter by tags**
field to match specific patterns. Trace Analyzer applies all filters together.

## View incoming traces

Trace Analyzer displays a **Tags** table and a **Tag values** table. By default,
each table displays up to 100 rows and calculates statistics from a rolling
15-second window. Select a numeric column heading to sort its values.

### Tags table

The **Tags** table displays available tag keys, how often each key appears, an
estimate of its unique values, and the size of its data. Click a tag key to add it
to the **Group by tags** field. The table contains these columns:

* **Tag key**: The tag key.
* **Appears in**: The percentage of spans in the rolling window that contain the
  tag key. Use this column to find tags that provide a broad breakdown of spans.
* **Unique value**: The estimated number of unique values for the tag key. Use this
  column to identify tags with many distinct values.
* **Total size (bytes)**: The total size of the tag data in the rolling window.
* **Avg. size (bytes)**: The average size of the tag data.

For example, consider the following spans:

```text theme={null}
span1: __service__ = "sign_up", location = "loc1", version = "0.100"
span2: __service__ = "sign_up", location = "loc2", version = "0.98"
span3: __service__ = "login", version = "0.98"
```

These spans generate three rows because the example contains three tag keys:
`__service__`, `location`, and `version`. All three spans contain the `__service__`
and `version` tag keys, so **Appears in** displays 100% for those rows. Two of the
three spans contain `location`, which displays as 67% after rounding.

Each tag key has two unique values, so **Unique value** displays 2 for every row.

The following table shows the statistics generated by the example spans:

| Tag key | Appears in | Unique value | Total size (bytes) | Avg. size (bytes) |
| - | - | - | - | - |
| `__service__` | 100% | 2 | 59,718,131 | 45 |
| `location` | 67% | 2 | 59,873,486 | 45 |
| `version` | 100% | 2 | 29,982,052 | 22 |

### Tag values table

The **Tag values** table groups spans by the tag keys selected in the
**Group by tags** field. The selected tag keys appear as the first columns and in
the order selected. Click a tag value to add it to the **Filter by tags** field.
The table contains these additional columns:

* **Avg. spans/sec**: The average rate of spans processed each second during the
  rolling window.
* **Total size (bytes)**: The total size of the grouped spans in the rolling window.
* **Avg. size (bytes)**: The average size of the grouped spans.

The following table shows example statistics when grouping by `__service__`:

| `__service__` | Avg. spans/sec | Total size (bytes) | Avg. size (bytes) |
| - | - | - | - |
| `ordering-svc` | 331 | 2,218,398 | 445 |
| `auth-svc` | 355 | 1,709,645 | 320 |
| `inventory-svc` | 331 | 2,218,398 | 447 |

## Promote tags for Cortex XCOR Differential Diagnosis (DDx)

When viewing a stream of incoming traces, the [tags table](#tags-table) displays all
tags related to your trace data. If you pause the live stream of traces, you can
select individual tags and promote them to top tags in the
[**Differential Diagnosis**](/investigate/analyze/differential-diagnosis/traces)
tab of Trace Explorer. Promoting tags makes them display in all differential
diagnosis results for every user in your Cortex XCOR tenant.

You must have administrative privileges to complete this task.

To promote tags to top tags:

1. In the navigation menu, click **<Icon icon="shield-user" /> Go to Admin**
   and then select
   **<MicroscopeIcon /> Analyzers <span aria-label="and then">></span> Live Telemetry**.

2. Click the **Traces** tab.

3. Click **Capture live data** to start profiling traces.

4. Click **Capture live data** again to pause the live stream of traces.

5. Hold the pointer over the tag you want to promote, click the <Icon icon="ellipsis-vertical" />
   three vertical dots icon, and then click **Add to top tags**.

   If the tag has more than 250 unique values, a confirmation dialog appears. Click
   **Add** to confirm.

   Cortex XCOR adds the tag to the tenant's top tags. Trace Explorer then
   includes the tag in results on the **Differential Diagnosis** tab. See
   [Access and use DDx](/investigate/analyze/differential-diagnosis/traces#access-and-use-ddx)
   for more information.

6. Complete the previous step to promote additional tags.

## Group and filter spans

Trace Analyzer groups spans by `__service__` and displays ingested spans by default.
Use the controls in these ways:

* In **Group by tags**, select up to 10 tag keys. The **Tag values** table adds a
  column for each key in the order selected. Clicking a key in the **Tags** table
  also adds it to this field.
* In **Filter by tags**, enter `key:value` pairs. For example, enter
  `service.name:traceingester`. Clicking a value in the **Tag values** table also
  adds it to this field.
* In **Data phase**, select one or more span classifications:
  * **Ingested**: Spans observed as they arrive during trace processing.
  * **Dropped**: Spans discarded by sampling decisions or product limits.
  * **Missing root**: Spans from traces that don't contain a root span.


## Related topics

- [Analyze live traffic metrics](/investigate/analyze/telemetry-analyzer.md)
- [Analyze data](/investigate/analyze.md)
- [Verify the Collector is receiving traces](/ingest/metrics-traces/collector/monitor/verify-traces.md)
- [Verify the Collector is scraping metrics](/ingest/metrics-traces/collector/monitor/verify-metrics.md)
- [Analyze metrics usage](/investigate/analyze/usage.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.