Repositories

A repository in TestRelic represents a single test project — typically one Git repository containing one or more test suites. The All Repos page (/entities) is your central hub for managing repository connections.

All Repos page

The All Repos page displays every repository connected to your organization as a card or table row, showing:

• Repository name and description
• Health status — An at-a-glance indicator of recent test health
• Pass rate — Percentage of passing tests in the most recent runs
• Trend — Pass rate direction (improving, stable, or degrading)
• Integration badges — Icons indicating which integrations are connected (GitHub Actions, Jira, etc.)
• Framework label — The test framework configured for this repo

Adding a repository

1. Click + Add Repository (the AddRepoDrawer button) on the All Repos page.
2. Enter a repository name and optional description.
3. Copy the generated API key — this key is placed in your reporter configuration.
4. Configure your reporter with the key:

For Playwright — add to playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['@testrelic/playwright-analytics', {
      apiKey: 'YOUR_API_KEY',
      repoId: 'YOUR_REPO_ID',
    }]
  ],
});

For all Playwright reporter options, see the Playwright SDK.

For Appium / WDIO — add to wdio.conf.ts:

import { TestRelicReporter } from '@testrelic/appium-analytics';

reporters: [
  'spec',
  [TestRelicReporter, {
    outputPath: './test-results/testrelic-timeline.json',
    cloud: {
      apiKey: 'YOUR_API_KEY',
    },
  }],
],

For all Appium reporter options, see the Appium SDK.

Repository detail

Click any repository to open the Repository Detail page. It opens on the Overview tab. The other tabs are Test Cases, Navigations, Test Runs, Test Plans, AI Agents testing (for repositories with eval data), Monitoring, Memory, Integrations, Alerts and Settings.

Overview tab

The Overview answers whether this repository is ready to release, and what stands in the way. It reads the same date range as the Test Runs tab's date filter. The line under the view switcher shows that range, the number of tests and the number of runs; when the range holds more than 100 runs, only the latest 100 are counted.

Pick a view with the switcher at the top:

Release readiness

The top of the Summary view gives a readiness score out of 100 and a verdict:

• Ship-ready: the score is 85 or more and nothing is blocking.
• Hold: the score is below 85. The verdict says how many points short it is.
• Blocked: a test in one of your critical paths is failing or flaky, whatever the score.

The score combines four parts. A part with nothing to measure, such as Critical paths before you choose any, is left out rather than scored as zero.

What moves the score lists the fixes that would raise the score most, with the score each one would reach. Ask AI for a fix plan opens Ask AI with a request for a plan to make the repository ship-ready.

Key numbers

A row of tiles gives the headline figures: Pass rate, Failing tests, Flaky tests (with the flake rate), Time to feedback (the median run time, with the p95), Time to recover and Runs.

Widgets

Besides the charts and lists the Overview already had, three widgets cover a release:

• Since the last run: new failures, fixes, still-failing tests and new tests in the latest run, compared with the run before it on the same branch.
• Work to green: what stands between the repository and a green release, ranked by what each fix unblocks, with each test's recent runs.
• Where the time goes: each suite's share of the total test time.

Pass rate trend leaves days with no runs as gaps rather than drawing them as 0%.

Filter, Customize and Share snapshot

• Filter opens the filter bar. Each filter you apply shows as a chip that you can remove on its own.
• Customize lets you rearrange the current view: drag a card by its handle, or use the card's menu to change its width, move it or remove it. Changes save to that view only. Add a widget offers the catalogue, or you can describe the widget you want to AI.
• Share snapshot creates a link to a snapshot of the Overview that anyone with the link can open. Choose when the link expires and, optionally, set a password. You can revoke a link at any time.

Overview settings

Open Overview settings from the More options (⋯) menu, from Choose critical paths (or Review critical paths) in the readiness breakdown, or from Edit targets on a scorecard. It's a full page with six sections:

While you edit, a preview re-scores readiness and the scorecards. Nothing changes on the Overview until you click Save changes. Discard drops your edits, and leaving with unsaved changes asks first (Discard and leave). Back to Overview returns to the dashboard.

Test Cases tab

Lists all unique test cases (by name and file path) ever recorded for this repository. Useful for understanding test suite coverage and identifying tests that have never passed.

Navigations tab

Embeds the Test Navigation (Pathfinder) view scoped to this specific repository, showing the navigation graph of browser flows captured across all runs.

Test Runs tab

Shows the run history for this repository. Equivalent to the org-wide Test Runs dashboard but pre-filtered to this repo.

AI Agents testing tab

Lists the LLM-evaluation runs recorded for this repository. Click through to per-case detail and per-case Test History, the same way you would for a test case. This tab appears for repositories that contain eval data. See LLM Evaluations for the full evaluation workflow.

Memory tab

The Memory tab holds durable, repository-scoped long-term memory — context that persists across sessions so AI assistants always start with precise, repo-specific knowledge instead of rediscovering it each time.

• Auto-categorized — saved memories are organized automatically by topic, so the memory stays browsable as it grows.
• Save from a conversation — capture useful context straight from an Ask AI conversation into the repository's memory with one action.
• Powers assistants — this memory backs the MCP server's memory tools and the command-line map command, giving both the same repo context. See the MCP server and the command-line interface.

Memory is scoped to a single repository, so context from one repo never bleeds into another.

Integrations tab

Displays the integration status for this repository. For each connected integration you can see the connection state and configure repository-level settings where the product supports it — for example GitHub Actions, Jira, Amplitude, and (when connected at the organization level) TestMu AI and BrowserStack Cloud Farms build filters and caching options.

Framework setup snippets

Inside the Repository Detail page, TestRelic displays setup code snippets specific to your configured framework.

Playwright snippets include:

• The npm install command for @testrelic/playwright-analytics
• A complete playwright.config.ts snippet with the reporter configured
• A link to the Playwright SDK guide

Appium snippets include:

• The npm install command for @testrelic/appium-analytics and its peer dependencies
• A complete wdio.conf.ts snippet with TestRelicService and TestRelicReporter configured
• A link to the Appium SDK guide

Ask AI from a repository

The Repository Detail page includes a direct link to open the AI assistant pre-loaded with context about that specific repository. This allows you to ask questions like "What are the most flaky tests in this repo?" with the right context already attached.

See Ask AI for more on AI context attachment.

Health metrics

Repository health is computed from recent run data:

• Pass rate — Percentage of test cases that passed in the most recent run. A flaky test, one that passed on a retry, counts as passed; see how pass rate is counted.
• Trend — Computed from the last 5 runs to indicate if health is improving or degrading.
• Flakiness score — Based on tests that alternate between pass and fail across runs.
• Eval Stability — For repositories with eval data, tracks how consistently eval cases pass across runs over time. See LLM Evaluations.

These metrics update automatically as new runs are uploaded. For a release decision, use the Overview tab's readiness score and verdict.