Ostra in your coding agent

Connect Claude Code, Cursor or VS Code to the Ostra MCP server, then teach your agent when to triage a failure, pick impacted tests, check selectors and ask Ostra Pro — with ready-made CLAUDE.md and Cursor rules.

Claude Code, Cursor, VS Code
Bearer API key
Ostra MCP server
ostra_triage_failure
ostra_select_impacted_tests
ostra_check_selector
ostra_pro_chat

With the Ostra MCP server connected, your coding agent can ask Ostra the same questions your pipeline does, while it works: is this failure flaky or real? which tests should I run for this change? will this locator survive? how do I fix this test? Tool calls are billed and rate limited exactly like the REST API.

1. Connect the server

Use a Full access key (read-only keys can list the tools but not call them). Keep the key in the OSTRA_API_KEY environment variable.

claude mcp add --transport http ostra https://ostra.testrelic.ai/mcp --header "Authorization: Bearer $OSTRA_API_KEY"

On Windows PowerShell use $env:OSTRA_API_KEY. Keep the command on one line. Check with claude mcp list: ostra should show ✓ Connected.

Ask the agent: "Call ostra_list_models and tell me the Ostra Decision release." If it answers, you're connected. Other clients and troubleshooting: MCP server.

2. Tell the agent when to use each tool

Agents use tools better when the project says when to. Add a short section to your agent instructions.

CLAUDE.md
## Ostra (testing models, via the `ostra` MCP server)

- When a test fails, call `ostra_triage_failure` before changing code. Pass the test id, the raw
  error message, `framework`, the failing `action`, and any run history you can find
  (`memory.runs_50`, `failed_count_50`, `flake_rate_50`). Then:
  - `flaky` / `likely_flaky`: don't change app code; suggest a retry or quarantine.
  - `product_bug`: investigate the application code, not the test.
  - `test_bug` / `selector_drift`: fix the test or the locator.
  - `environment`: fix the setup (browsers, dependencies, ports), not the code.
  - `defer`: say the evidence is inconclusive and show what you found.
- Before running the whole suite after a change, call `ostra_select_impacted_tests` with the diff
  stats and the test files as candidates; run `output.impacted` first.
- When adding or changing locators, call `ostra_check_selector`; replace any that come back `breaks`
  with role or test-id locators.
- For hard test-writing or failure-analysis questions, ask `ostra_pro_chat`, passing the relevant
  code and logs in the prompt and the same `session` value for follow-ups on the same problem.
- Don't call Ostra tools in loops over unchanged inputs: identical requests within 24 hours return
  the cached answer.

Example workflows

"The checkout test is failing again. What's going on?" The agent reads the last failure, calls ostra_triage_failure, gets flaky at 83%, and suggests quarantining the test and stabilising the wait instead of editing the checkout code.

"I changed the coupon service. Which tests should I run?" The agent summarises git diff --numstat, lists the spec files as candidates (with co-change rates from git log if it can), calls ostra_select_impacted_tests, and runs the two checkout specs it returns.

"Write a Playwright test for password reset." The agent asks ostra_pro_chat with the page's markup and your existing test style, writes the test, then calls ostra_check_selector on the locators it used and swaps any that come back breaks.

Cost and control

  • Every call shows up in the Console under MCP servers → Recent tool calls and Logs (channel MCP), with its cost and the client that made it.
  • Owners, admins and developers can turn any tool off for the workspace under MCP servers → Tools.
  • Give agents their own key with a monthly spend cap, so an agent stuck in a loop can't spend more than you allow. See API keys.
Was this page helpful?

On this page