> ## Documentation Index
> Fetch the complete documentation index at: https://docs-xcor.paloaltonetworks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# BlazeMeter API Monitoring

> BlazeMeter API Monitoring (formerly Runscope) test results from the API Monitoring API, one value per finished test run and per request step, such as pass or fail, assertion, script, and variable results, response times, status codes, and response sizes, under the metric names Runscope sends to Datadog.

The BlazeMeter API Monitoring integration connects Cortex XCOR with
[BlazeMeter](https://www.blazemeter.com/) API Monitoring, the product formerly called Runscope.
BlazeMeter API Monitoring runs your API tests on a schedule from its cloud locations or from your
own Radar agents. XCOR collects every passing and failing test run in the buckets your
access token can see. The collected results include pass or fail, assertion, script, and variable
results, response times, status codes, and response sizes for each request step.

## Supported telemetry types

The BlazeMeter API Monitoring integration supports these telemetry types:

| Logs | Metrics | Traces | Events |
| :-: | :-: | :-: | :-: |
| No | Yes | No | No |

## Compatibility

The BlazeMeter API Monitoring integration works with:

* BlazeMeter API Monitoring tests, through the API Monitoring API at `https://api.runscope.com`.
* Every bucket the access token's user can see, unless you list the buckets to collect or to
  skip. Buckets added later are picked up automatically, unless you list the buckets to collect.
* Runs from cloud locations and from Radar agents.
* Every test in the buckets one configuration collects.

XCOR reads each collected test's results once a minute and lists your buckets, tests, and
test steps every five minutes, using your access token.

## Prerequisites

The BlazeMeter API Monitoring integration requires:

* A BlazeMeter API Monitoring access token. Creating an API application in BlazeMeter API
  Monitoring issues an access token for your own account. For details, see
  [Authentication](https://help.blazemeter.com/apidocs/api-monitoring/authentication.htm) in the
  BlazeMeter API Monitoring API documentation.
* The `api:read`, `test:read`, and `team:read` scopes on the token. BlazeMeter doesn't document
  which scopes the access token it issues for your own account carries.

The access token provides read access to every bucket its user can see. If the token carries write
scopes, it can also make changes, although the integration only reads. Create the API application
as a dedicated BlazeMeter API Monitoring user that belongs to only the teams whose buckets you
want collected.

## Configure the integration

1. In BlazeMeter API Monitoring, sign in as the dedicated user, create an API application, and
   copy the access token that BlazeMeter issues for your own account.
2. Decide which buckets to collect. Leave both bucket lists empty to collect every bucket the
   token can see. To collect only some buckets, list their keys in **Buckets to collect**. To
   collect all but some, list their keys in **Buckets to skip**. Use one list or the other, not
   both. A bucket key is the identifier in the bucket's API URLs, not its name.
3. In XCOR, open the Integrations Hub and select **BlazeMeter API Monitoring**.
4. On the **Configurations** tab, add a configuration, enter a name, the access token, and your
   bucket list, and save.

A configuration collects every test in the buckets you select. XCOR paces the requests it
makes with each access token, and reads every detail of at least about 375 tests per configuration.
When a larger selection outruns that pace, the runs' results still come first: some step details
are skipped and some results can arrive late. Every BlazeMeter API Monitoring configuration in your
account shares the same collection capacity. Past about 520 tests in one configuration, or when two
configurations each hold more than about 375 tests, XCOR can fall behind. Then some runs
aren't collected, for every configuration and not only the large ones. Splitting a selection
across configurations or access tokens doesn't add that capacity. The larger the selection, the
longer each listing of your tests and steps takes, so new tests and steps can take longer to be
picked up. To collect every detail of a large selection, narrow it. Two configurations that share
a token spend its request rate twice as fast.

Give each access token one configuration, and don't let two configurations collect the same
bucket. Two configurations that can both see a bucket, through the same token or through users
who share a team, read its runs twice. This doubles the number of BlazeMeter API Monitoring
requests and can occasionally count a run twice. You can't save a second configuration that uses
the same connection as an existing one, but XCOR can't detect when you enter the same token
in two configurations.

### Save validation

When you save, XCOR checks the token's form and verifies the token by listing one bucket
in BlazeMeter API Monitoring. A failure returns one of these messages:

* ``Invalid value for field `access_token`: must not contain whitespace or control characters``:
  the pasted token includes a space or a line break. XCOR refuses it before calling
  BlazeMeter API Monitoring.
* `BlazeMeter API Monitoring access_token is not valid`: BlazeMeter API Monitoring rejected the
  token. Check that you copied the whole token and that it hasn't been revoked.
* `BlazeMeter API Monitoring refused the credential check (HTTP <code>)`: BlazeMeter API
  Monitoring refused the request for a reason other than the token.
* `BlazeMeter API Monitoring credential verification temporarily unavailable (HTTP <code>)`:
  BlazeMeter API Monitoring is throttling or unavailable. Try again later.
* `BlazeMeter API Monitoring credential verification got an unexpected answer (HTTP <code>)`, or
  `BlazeMeter API Monitoring credential verification failed: <reason>`: XCOR couldn't
  complete the check. Try again later, and contact Cortex XCOR Support if it persists.

You also can't save a configuration that lists buckets to collect and buckets to skip together.

Save doesn't validate the token's scopes or which buckets the token can see. A configuration with a
token whose user can't see any buckets saves, but collects nothing until the user can see a
bucket.

## Validate metrics ingestion

1. Allow up to about 20 minutes after saving for the first metrics to appear. XCOR starts
   collecting within about 10 minutes of saving, and runs that finish before then might not be
   collected. After that, make sure a test in a collected bucket finishes a run, and look for it
   about 7 minutes later.
2. On the **Configurations** tab, check the configuration's status. **OK** means the token works
   and can see every bucket in **Buckets to collect**. **Error** comes with a message, which
   Troubleshooting explains.
3. In [Metrics Explorer](https://docs-xcor.paloaltonetworks.com/investigate/querying/metrics/explorer.md),
   count the runs collected in the last 30 minutes. Each run writes one data point, stamped at the
   run's finish time, so use a range rather than the current instant:

   ```text theme={null}
   sum(count_over_time(RunscopeTestRun_test_failed[30m]))
   ```

   A result means data is flowing. The newest data point is about 5 minutes old or older, so for
   tests that run less often than every 20 minutes, widen the range. To filter for one bucket, add
   its key, for example `RunscopeTestRun_test_failed{bucket_key="<your bucket key>"}`.

## Dashboards

The integration includes an **Overview** dashboard that shows failing tests, failed runs and pass
rate by bucket, passed and failed runs by test, response times by test and request step, failed
assertions, the share of request steps collected without step detail, and collection volume. Open
it from the **Assets** tab on the BlazeMeter API Monitoring integration page in the Integrations
Hub. For more about bundled dashboards, see
[Use bundled assets](https://docs-xcor.paloaltonetworks.com/integrate/integrations-hub.md#use-bundled-assets).

## Metric semantics

The 13 `RunscopeTestRun_*` metrics hold one value for each run, and the 12 `RunscopeTestRequest_*`
metrics hold one value for each request step of a run. They're the metric names Runscope sends to
Datadog, so queries ported from Datadog keep their metric names and label names, except
`agent_expired` and the requested URL tag on request metrics, which the integration doesn't
carry. Label values are normalized the way Datadog
normalizes Runscope's tags, so a ported filter on `bucket`, `test_name`, or `environment` matches
the same value. One value differs by design: run metrics' `status` reads `pass` or `fail` where
Runscope sends `passed` or `failed`, so change a ported filter such as `status:passed` to
`status="pass"`. The names contain no dots, so plain selectors such as
`RunscopeTestRun_test_failed{bucket="payments"}` work.

Every series carries `bucket`, `test_name`, `bucket_key`, and `test_uuid`. It can also carry
`team`, `environment`, and `environment_uuid`. A run from a cloud location
carries `region` and `region_name`, the location's name, such as `us_virginia`, and a run from a
Radar agent carries `agent`. Run metrics add
`status`, which reads `pass` or `fail`. Request metrics add `request_method`, `note`, and
`step_index`, which records the step's position in the run as a value from 1 to 100. Runscope's
Datadog metrics have no `step_index`, so aggregate it away. The Metrics tab lists every label.

XCOR normalizes provider label values the way Datadog normalizes tag values, including
`bucket_key` and `test_uuid`, so filter on the lowercased form. Each value is lowercased, every
character other than a letter, digit, `-`, `:`, `.`, or `/` becomes `_`, repeated underscores
collapse to one, and a trailing underscore is removed. A leading underscore stays, as it does in
Datadog. For example, a bucket named `Payments API (v2)` reads `payments_api_v2`, and a step note
that starts `[Orders] Create` reads `_orders_create`. Every series also carries
`{"xcor.integration.name"="blazemeterapimonitoring"}`.

Because a series has a data point only when a run finishes:

* Wrap every query in a range function, such as `sum_over_time`, `avg_over_time`, or
  `max_over_time`, over a range at least as long as the time between runs plus about 10 minutes.
  A plain selector rarely shows data.
* Count failed runs with `sum(sum_over_time(RunscopeTestRun_test_failed[30m]))`. Each run writes
  `RunscopeTestRun_test_failed` once, as 1 for a failed run and 0 for a passed run, so the failure
  ratio is
  `sum(sum_over_time(RunscopeTestRun_test_failed[1h])) / sum(count_over_time(RunscopeTestRun_test_failed[1h]))`.
* The average of `RunscopeTestRun_test_failed` across series isn't the failure ratio. Each value
  of `status` is its own series, so the average weighs series, not runs. Datadog shows the same
  average for the same query.
* Each data point arrives about 5 to 7 minutes after its run finishes. A query or monitor over the
  last 5 minutes can come back empty even when a run failed in that time, so use a range of at
  least 10 minutes.
* After long-term downsampling, about five days after ingest, XCOR keeps the last data
  point of each series in each five-minute interval. Over older ranges, run and failure counts
  undercount a test that finishes more than one run with the same result, location, and
  environment within five minutes, and failure ratios can read higher or lower than they were,
  usually higher for a test that mostly passes.

For the full metric catalog, open the **Metrics** tab on the BlazeMeter API Monitoring integration
page in the Integrations Hub.

## Troubleshooting

The following table lists symptoms and how to resolve them:

| Symptom | Resolution |
| - | - |
| The configuration shows **Error** with the message `Invalid credentials. Update the credentials in this configuration.` or `Credentials were rejected. Check that they are valid and have the permissions listed in the setup guide.` | BlazeMeter API Monitoring refused the token, for example because it was revoked. The configuration stops collecting about 15 minutes later. Update the configuration with a new access token, and XCOR resumes collecting within about 10 minutes. Runs that finish before you replace the token aren't collected. |
| The configuration shows **Error** with the message `This configuration's details don't match the account. Correct the configuration to receive data.` | **Buckets to collect** lists a key the token can't see. The other buckets are still collected. Correct the keys in **Buckets to collect**, or add the token's user to the teams that own those buckets. |
| Save succeeds, but no metrics appear. | Check that the token's user belongs to the teams that own the buckets, and that a test in a collected bucket ran and passed or failed. Runs that are canceled, or that take longer than 15 minutes, aren't collected. |
| `RunscopeTestRequest_response_status_code`, `RunscopeTestRequest_response_time_ms`, or `RunscopeTestRequest_response_size_bytes` is missing for some steps. | A step that received no response, for example because its request timed out, has none of the three, where Runscope sent a response time of -1 or -2 and no status code. Its run's `RunscopeTestRun_total_response_time` adds up only the steps that received a response, as Runscope's did. The status code and response size also come from each step's detail, which BlazeMeter API Monitoring keeps only for the newest 100 passing and 100 failing runs of each test. XCOR also skips step details when reading them would exceed the request rate it allows itself per token, which a large selection can. |
| A monitor ported from Datadog with a 5-minute window doesn't fire. | Data points arrive about 5 to 7 minutes after each run finishes. Widen the monitor's range to at least 10 minutes. |
| Some runs are counted twice. | Two configurations collect the same buckets, and for a while after a rename, for example of a bucket or test, their copies of a run carry different label values. Remove the overlap, so that one configuration collects each bucket. |

## Limitations

* Only runs that pass or fail are collected, once they finish. Runs that are still running or
  queued, canceled runs, and runs that take longer than 15 minutes from start to finish aren't
  collected.
* Only a test's own HTTP request steps are collected, up to entry 100 in a run. Requests
  that a subtest step makes aren't collected. If a collected request step occurs after entry 100,
  `RunscopeTestRun_total_response_time` is absent.
* Response times might not match the values Runscope sent to Datadog for the same run.
  XCOR takes a step's response time from the step's detail when BlazeMeter API Monitoring returns
  it, and otherwise adds up the step's timings, so the source can vary from run to run. Without
  the step's detail, a step that received no response can't be told apart, so its response time
  is the time it waited.
* Runscope's per-run Datadog events, its request URL tag, and its data iteration metrics aren't
  collected. The integration collects no logs, traces, or events, and uses no webhooks.
* Run history from before XCOR starts collecting isn't collected. After an interruption on
  the XCOR side, runs that finished more than about an hour before collecting resumed
  aren't collected. Runs that finish while BlazeMeter API Monitoring rejects the token, or
  throttles it past the waits XCOR allows, aren't collected later. Neither are runs that
  finish while their bucket isn't selected.
* Each data point arrives several minutes after its run finishes, which is later than many rollup
  rules accept, so a rollup rule on these metrics can miss data points. When XCOR answers
  an aggregate query from such a rule, a query like
  `sum(sum_over_time(RunscopeTestRun_test_failed[30m]))` can come back lower than the stored data.
* Labels set in a configuration's `metric_labels` field through the Cortex XCOR API aren't added
  to these metrics.
* XCOR reads only the public API at `https://api.runscope.com`. A private or regional API
  host isn't supported.


## Related topics

- [Server integrations](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/server.md)
- [MarkLogic](https://docs-xcor.paloaltonetworks.com/ingest/xcor/integrations/collector/marklogic.md)
- [Data API](https://docs-xcor.paloaltonetworks.com/tooling/api-info/data_definition/index.md)
- [State API](https://docs-xcor.paloaltonetworks.com/tooling/api-info/state_definition/index.md)
- [Config API](https://docs-xcor.paloaltonetworks.com/tooling/api-info/definition/index.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.