# Ostra API reference (TestRelic) Ostra is TestRelic’s family of testing models. Ostra Decision (API id `ostra-lite`, beta) answers three CI questions: failure triage, test impact and selector survival. Ostra Pro (`ostra-pro`, 1.0) is a testing model for writing and fixing tests, analysing failures and logs, and reading screenshots and video, with OpenAI- and Anthropic-compatible endpoints. With an API key both are pay as you go and prepaid: add credit in the TestRelic Console (https://console.testrelic.ai) and create an API key there. The Growth, Pro and Enterprise plans also include Ostra usage (https://testrelic.ai/pricing). - REST base URL: `https://ostra.testrelic.ai/api/v1` - MCP server: `https://ostra.testrelic.ai/mcp` (Streamable HTTP) - Auth: `Authorization: Bearer ` (Ostra Pro routes also take `x-api-key`). Keys start with `ostk_live_` and are created in the TestRelic Console (https://console.testrelic.ai/keys). - Errors: `{"error": {"type", "message", "request_id", ...}}`; Anthropic shape on `/messages`. - OpenAPI: https://docs.testrelic.ai/lores/api/openapi.json # Endpoints ## List models `GET https://ostra.testrelic.ai/api/v1/models` Lists the models your key can call, with their tasks, limits and prices. Use it to check a key works (any key, read-only included) and to read the live Ostra Decision release. It’s free and not rate limited. - **Auth:** `Authorization: Bearer `; any key. - **Rate limited:** no. - **Cost:** Free. ### Response | Field | Type | Required | Description | |---|---|---|---| | `object` | string | No | Always `list`. One of `list`. | | `data` | object[] | No | One entry per model: Ostra Decision, then Ostra Pro. | | `data[].id` | string | No | Model id to send as `model`: `ostra-lite` (Ostra Decision) or `ostra-pro` (Ostra Pro). | | `data[].name` | string | No | Display name: `Ostra Decision` or `Ostra Pro`. | | `data[].status` | string | No | Ostra Decision only: `beta`. | | `data[].release` | string | No | Ostra Decision only: the live release, e.g. `v.beta.5`, or `unavailable` if it can’t be reached right now. | | `data[].tasks` | object[] | No | Ostra Decision only: its three tasks. | | `data[].tasks[].task` | string | No | Task id to send as `task`. One of `triage`, `test_impact`, `selector_survival`. | | `data[].tasks[].name` | string | No | Display name. | | `data[].tasks[].description` | string | No | What the task answers. | | `data[].tasks[].mcpTool` | string | No | The MCP tool that runs this task. | | `data[].tasks[].rateLabel` | string | No | Input price, e.g. `$0.50 / 1M failures`. | | `data[].tasks[].outputRateLabel` | string | No | Output price (always `$0.00`). | | `data[].tasks[].version` | string | No | Version of this task’s model in the live release. | | `data[].tasks[].contentHash` | string | No | Content hash of that model. | | `data[].tasks[].gatePassed` | boolean | No | The model passed its release quality gate. | | `data[].tasks[].classes` | string[] | No | The classes the task can answer. | | `data[].tasks[].deferThreshold` | number | No | The task’s default confidence bar for a firm answer. A class can use a different bar; each answer reports the one it used in `threshold`. | | `data[].pricing` | map | No | Human-readable prices: `input`, `output`, and for Ostra Pro `cached_input`. | | `data[].latencyP50Ms` | number \| null | No | Ostra Decision only: your workspace’s median latency over 7 days. | | `data[].requests7d` | integer | No | Ostra Decision only: your workspace’s requests over 7 days. | | `data[].object` | string | No | Ostra Pro only: `model`. | | `data[].created` | integer | No | Ostra Pro only: Unix time the model version was released. | | `data[].owned_by` | string | No | Ostra Pro only: `testrelic`. | | `data[].version` | string | No | Ostra Pro only: `1.0`. | | `data[].base_url` | string | No | Ostra Pro only: base URL for OpenAI-compatible clients. | | `data[].context_window` | integer | No | Ostra Pro only: 262,144 tokens. | | `data[].max_output_tokens` | integer | No | Ostra Pro only: 32,768 tokens. | | `data[].mcp_tool` | string | No | Ostra Pro only: `ostra_pro_chat`. | ### Response headers | Field | Type | Required | Description | |---|---|---|---| | `x-request-id` | string | No | Id of this request (`req_` + 26 characters). Quote it to support. | ### Errors | Status | Type | When | |---|---|---| | 401 | `authentication_error` | The API key is missing, malformed, unknown, revoked or expired. | | 403 | `permission_error` | The caller’s IP isn’t in the key’s allowlist. | ### Example: List models ```bash curl https://ostra.testrelic.ai/api/v1/models \ -H "Authorization: Bearer $OSTRA_API_KEY" ``` ### Example response (200 · Models) ```json { "object": "list", "data": [ { "id": "ostra-lite", "name": "Ostra Decision", "status": "beta", "release": "v.beta.5", "tasks": [ { "task": "triage", "name": "Failure triage", "description": "Classifies a failing test as flaky, product bug, test bug, environment or selector drift.", "mcpTool": "ostra_triage_failure", "rateLabel": "$0.50 / 1M failures", "outputRateLabel": "$0.00 / 1M decisions", "version": "1.4.0", "contentHash": "sha256:3f9a1c07d2e4b8a6", "gatePassed": true, "classes": [ "flaky", "product_bug", "test_bug", "environment", "selector_drift" ], "deferThreshold": 0.6 }, { "task": "test_impact", "name": "Test impact", "description": "Ranks candidate tests by the probability that a diff breaks them.", "mcpTool": "ostra_select_impacted_tests", "rateLabel": "$2.00 / 1M diffs", "outputRateLabel": "$0.00 / 1M decisions", "version": "1.1.2", "contentHash": "sha256:a07be3315c9d4f12", "gatePassed": true, "classes": [ "not_impacted", "impacted" ], "deferThreshold": 0.55 }, { "task": "selector_survival", "name": "Selector survival", "description": "Predicts whether UI locators survive the next release.", "mcpTool": "ostra_check_selector", "rateLabel": "$0.25 / 1M selectors", "outputRateLabel": "$0.00 / 1M decisions", "version": "1.2.0", "contentHash": "sha256:5d21f0e9c4a7b3e8", "gatePassed": true, "classes": [ "survives", "breaks" ], "deferThreshold": 0.6 } ], "pricing": { "input": "$0.25–$2.00 / 1M inputs", "output": "$0.00 / 1M decisions" }, "latencyP50Ms": 41, "requests7d": 1284 }, { "id": "ostra-pro", "object": "model", "created": 1794873600, "owned_by": "testrelic", "name": "Ostra Pro", "version": "1.0", "base_url": "https://ostra.testrelic.ai/api/v1", "context_window": 262144, "max_output_tokens": 32768, "pricing": { "input": "$0.64 / 1M input tokens", "cached_input": "$0.06 / 1M cached input tokens", "output": "$4.47 / 1M output tokens" }, "mcp_tool": "ostra_pro_chat" } ] } ``` Reference page: https://docs.testrelic.ai/lores/api/models/list ## Run a task `POST https://ostra.testrelic.ai/api/v1/predict` Runs one Ostra Decision task (failure triage, test impact or selector survival) and returns its decision. Send `model: "ostra-lite"`, the `task`, and that task’s `input`. The answer is in `output`, in the shape for the task. An identical task and input within 24 hours is answered from cache, for free. - **Auth:** `Authorization: Bearer `; Full-access key. - **Rate limited:** yes (counts against the workspace’s requests per minute). - **Cost:** Triage $0.50 per 1M failures; test impact $2.00 per 1M requests; selector survival $0.25 per 1M locators. Output is free. Cached answers and errors cost nothing; a `defer` answer is billed. ### Request body | Field | Type | Required | Description | |---|---|---|---| | `model` | string | Yes | Always `ostra-lite` (Ostra Decision). One of `ostra-lite`. | | `task` | string | Yes | Which task to run. One of `triage`, `test_impact`, `selector_survival`. | | `input` | object | Yes | The task’s input. Unknown fields are rejected. See the three task shapes below. | ### `input` for `triage` (Failure triage) | Field | Type | Required | Description | |---|---|---|---| | `test_id` | string | No | Your identifier for the failing test, echoed back in the answer. Use a stable name, e.g. `checkout.spec.ts > applies coupon`. 1–512 characters. | | `run_id` | string | No | Your identifier for the CI run, echoed back. 1–512 characters. | | `context` | object | Yes | Evidence about the failure. Every field is optional except that `error_class` or `error_message` must be present. Missing evidence is treated as missing, never as zero, so send what you have. | | `context.id` | string | No | Your own id for this failure record. 1–512 characters. | | `context.framework` | string | No | Test framework that produced the failure. One of `playwright`, `cypress`, `jest`, `selenium`. | | `context.capture_level` | string | No | How much of the failure was captured (trace, DOM, network): `full`, `partial` or `none`. One of `full`, `partial`, `none`. | | `context.error_class` | string | No | Normalized class of the error. Required unless you send `error_message`. One of `timeout`, `locator_not_found`, `stale_element`, `strict_mode_violation`, `assertion_mismatch`, `network_error`, `script_error`, `unmapped`. | | `context.error_message` | string | No | The raw error text. Used only to derive `error_class` when you don’t send one; the answer reports which you used in `error_class_source`. 0–8000 characters. | | `context.action` | string | No | The step that failed. One of `click`, `fill`, `navigate`, `assert`, `wait_for`, `api_call`, `screenshot`. | | `context.locator` | object | No | The locator the failing step used. | | `context.locator.strategy` | string | No | How the element was located. One of `css`, `xpath`, `test_id`, `text`, `role`, `none`. | | `context.diff` | object | No | The code change the run tested. | | `context.diff.files_changed` | integer | No | Files changed in the commit or pull request under test. Range 0–1000000. | | `context.diff.lines_added` | integer | No | Lines added. Range 0–1000000. | | `context.diff.lines_removed` | integer | No | Lines removed. Range 0–1000000. | | `context.diff.touches_test_file` | boolean | No | The change edits a test file. | | `context.diff.touches_selector_file` | boolean | No | The change edits a page object, locator map or other file that defines selectors. | | `context.diff.locator_values_removed` | boolean | No | The change removes or renames a locator value (an id, test id, class or text) that the test uses. | | `context.dom` | object | No | What the page looked like at failure time. | | `context.dom.locator_found` | boolean | No | The locator still matched an element in the captured DOM. | | `context.memory` | object | No | This test’s recent history, over its last 50 runs at most. | | `context.memory.runs_50` | integer | No | Runs in the window. Range 0–50. | | `context.memory.failed_count_50` | integer | No | Failed runs in the window. Range 0–50. | | `context.memory.passed_on_retry_50` | integer | No | Runs that failed and then passed on retry. Range 0–50. | | `context.memory.consecutive_failures_50` | integer | No | Failures in a row up to this one. Range 0–50. | | `context.memory.flake_rate_50` | number | No | Share of runs in the window that were flaky, 0–1. Range 0–1. | | `context.memory.signature_repeat_count` | integer | No | How many times this exact error signature has been seen before. Range 0–1000000. | ### `output` for `triage` | Field | Type | Required | Description | |---|---|---|---| | `decision_id` | string | No | Unique id of this decision: `dec_` and 32 hex characters. | | `task` | string | No | The task that answered. One of `triage`, `test_impact`, `selector_survival`. | | `decision` | string | No | The answer: one of the task’s classes, `likely_flaky` (triage only), or `defer` when the model isn’t confident enough. | | `value` | string \| null | No | The same answer, or `null` when the decision is `defer`. | | `p` | number | No | Probability of the top class, 0–1. | | `probabilities` | map | No | Probability of every class, keyed by class. | | `threshold` | number | No | The confidence the top class must reach for a firm answer. Below it, the answer is `defer` (or `likely_flaky`). | | `defer_reason` | string \| null | No | Why the answer is `defer`; `null` otherwise. `insufficient_evidence` means the request carried nothing about this test or locator. One of `below_threshold`, `model_gate_failed`, `insufficient_evidence`. | | `decided_by` | string | No | `rule` when an unambiguous environment rule answered (see `rule`). The probabilities are still the model’s. One of `model`, `rule`. | | `rule` | string \| null | No | The rule that answered, when `decided_by` is `rule`. A rule answer is always `environment`. One of `browser_not_installed`, `host_missing_dependencies`, `webdriver_session_not_created`, `port_in_use`, `disk_full`. | | `model` | object | No | The model release that answered. | | `model.release` | string | No | Ostra Decision release, e.g. `v.beta.5`. | | `model.version` | string | No | Version of this task’s model in that release. | | `model.content_hash` | string | No | Content hash of the model that answered. | | `model.gate_passed` | boolean | No | The model passed its release quality gate. When `false`, every answer is `defer` with `model_gate_failed`. | | `test_id` | string \| null | No | Your `test_id`, echoed. | | `run_id` | string \| null | No | Your `run_id`, echoed. | | `error_class` | string | No | The error class the model used. One of `timeout`, `locator_not_found`, `stale_element`, `strict_mode_violation`, `assertion_mismatch`, `network_error`, `script_error`, `unmapped`. | | `error_class_source` | string | No | `request` when you sent `error_class`, `error_message` when it was derived from your message. One of `request`, `error_message`. | ### `input` for `test_impact` (Test impact) | Field | Type | Required | Description | |---|---|---|---| | `diff` | object | Yes | The code change to evaluate. | | `diff.files_changed` | integer | No | Files changed in the commit or pull request under test. Range 0–1000000. | | `diff.lines_added` | integer | No | Lines added. Range 0–1000000. | | `diff.lines_removed` | integer | No | Lines removed. Range 0–1000000. | | `diff.touches_test_file` | boolean | No | The change edits a test file. | | `diff.touches_selector_file` | boolean | No | The change edits a page object, locator map or other file that defines selectors. | | `candidates` | object[] | Yes | The tests that could be affected. One request is billed once, whatever the number of candidates. 1–2000 items. | | `candidates[].test_id` | string | Yes | Your identifier for the test. 1–512 characters. | | `candidates[].co_change_rate` | number | No | How often this test’s files change together with the changed files, 0–1. Range 0–1. | | `candidates[].historical_fail_rate` | number | No | The test’s historical failure rate, 0–1. Range 0–1. | ### `output` for `test_impact` | Field | Type | Required | Description | |---|---|---|---| | `decisions` | object[] | No | One decision per candidate (`impacted`, `not_impacted` or `defer`), sorted by probability of `impacted`, highest first. | | `decisions[].decision_id` | string | No | Unique id of this decision: `dec_` and 32 hex characters. | | `decisions[].task` | string | No | The task that answered. One of `triage`, `test_impact`, `selector_survival`. | | `decisions[].decision` | string | No | The answer: one of the task’s classes, `likely_flaky` (triage only), or `defer` when the model isn’t confident enough. | | `decisions[].value` | string \| null | No | The same answer, or `null` when the decision is `defer`. | | `decisions[].p` | number | No | Probability of the top class, 0–1. | | `decisions[].probabilities` | map | No | Probability of every class, keyed by class. | | `decisions[].threshold` | number | No | The confidence the top class must reach for a firm answer. Below it, the answer is `defer` (or `likely_flaky`). | | `decisions[].defer_reason` | string \| null | No | Why the answer is `defer`; `null` otherwise. `insufficient_evidence` means the request carried nothing about this test or locator. One of `below_threshold`, `model_gate_failed`, `insufficient_evidence`. | | `decisions[].decided_by` | string | No | `rule` when an unambiguous environment rule answered (see `rule`). The probabilities are still the model’s. One of `model`, `rule`. | | `decisions[].rule` | string \| null | No | The rule that answered, when `decided_by` is `rule`. A rule answer is always `environment`. One of `browser_not_installed`, `host_missing_dependencies`, `webdriver_session_not_created`, `port_in_use`, `disk_full`. | | `decisions[].model` | object | No | The model release that answered. | | `decisions[].model.release` | string | No | Ostra Decision release, e.g. `v.beta.5`. | | `decisions[].model.version` | string | No | Version of this task’s model in that release. | | `decisions[].model.content_hash` | string | No | Content hash of the model that answered. | | `decisions[].model.gate_passed` | boolean | No | The model passed its release quality gate. When `false`, every answer is `defer` with `model_gate_failed`. | | `decisions[].test_id` | string | No | The candidate’s `test_id`. | | `impacted` | string[] | No | The tests to run, best first. See `selection` for how they were chosen. | | `selection` | string | No | `model`: `impacted` holds the candidates answered `impacted`. `fallback_top_candidates`: none cleared the threshold, so up to 5 best-ranked candidates with evidence are returned. `none`: no candidate carries any evidence. One of `model`, `fallback_top_candidates`, `none`. | ### `input` for `selector_survival` (Selector survival) | Field | Type | Required | Description | |---|---|---|---| | `locators` | object[] | Yes | The locators to check. Billed per locator. 1–500 items. | | `locators[].locator_id` | string | No | Your identifier for the locator, echoed back. 1–512 characters. | | `locators[].strategy` | string | No | How the locator finds its element. One of `css`, `xpath`, `test_id`, `text`, `role`, `none`. | | `locators[].dom_locator_found` | boolean | No | The locator matches an element in the current build’s DOM. | | `locators[].history` | object[] | No | How well the locator resolved in past releases, oldest first. 0–100 items. | | `locators[].history[].release` | string | Yes | Release label, e.g. `2026.11.1`. 1–128 characters. | | `locators[].history[].resolve_rate` | number | No | Share of runs in that release where the locator resolved, 0–1. Range 0–1. | ### `output` for `selector_survival` | Field | Type | Required | Description | |---|---|---|---| | `decisions` | object[] | No | One decision per locator, in input order: `survives`, `breaks` or `defer`. | | `decisions[].decision_id` | string | No | Unique id of this decision: `dec_` and 32 hex characters. | | `decisions[].task` | string | No | The task that answered. One of `triage`, `test_impact`, `selector_survival`. | | `decisions[].decision` | string | No | The answer: one of the task’s classes, `likely_flaky` (triage only), or `defer` when the model isn’t confident enough. | | `decisions[].value` | string \| null | No | The same answer, or `null` when the decision is `defer`. | | `decisions[].p` | number | No | Probability of the top class, 0–1. | | `decisions[].probabilities` | map | No | Probability of every class, keyed by class. | | `decisions[].threshold` | number | No | The confidence the top class must reach for a firm answer. Below it, the answer is `defer` (or `likely_flaky`). | | `decisions[].defer_reason` | string \| null | No | Why the answer is `defer`; `null` otherwise. `insufficient_evidence` means the request carried nothing about this test or locator. One of `below_threshold`, `model_gate_failed`, `insufficient_evidence`. | | `decisions[].decided_by` | string | No | `rule` when an unambiguous environment rule answered (see `rule`). The probabilities are still the model’s. One of `model`, `rule`. | | `decisions[].rule` | string \| null | No | The rule that answered, when `decided_by` is `rule`. A rule answer is always `environment`. One of `browser_not_installed`, `host_missing_dependencies`, `webdriver_session_not_created`, `port_in_use`, `disk_full`. | | `decisions[].model` | object | No | The model release that answered. | | `decisions[].model.release` | string | No | Ostra Decision release, e.g. `v.beta.5`. | | `decisions[].model.version` | string | No | Version of this task’s model in that release. | | `decisions[].model.content_hash` | string | No | Content hash of the model that answered. | | `decisions[].model.gate_passed` | boolean | No | The model passed its release quality gate. When `false`, every answer is `defer` with `model_gate_failed`. | | `decisions[].locator_id` | string \| null | No | Your `locator_id`, echoed. | ### Response | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Request id, the same as the `x-request-id` header. | | `model` | string | No | The model and release that answered, e.g. `ostra-lite@v.beta.5`. | | `task` | string | No | The task you ran. | | `cached` | boolean | No | `true` when the answer came from the 24-hour cache (and cost nothing). | | `billed` | object | No | What this request cost. Cached answers cost 0. | | `billed.units` | integer | No | Billable units: 1 per failure (triage), 1 per request (test impact), 1 per locator (selector survival). 0 for a cached answer. | | `billed.costNano` | integer | No | Cost in nano-dollars (1 USD = 1,000,000,000). | | `billed.costUsd` | string | No | Cost in USD as an exact decimal string, e.g. `"0.0000005"`. | | `output` | object | No | The task’s answer. Shapes: triage, test impact and selector survival below. | ### Response headers | Field | Type | Required | Description | |---|---|---|---| | `x-request-id` | string | No | Id of this request (`req_` + 26 characters). Quote it to support. | | `ostra-ratelimit-requests-limit` | integer | No | Your workspace’s requests-per-minute limit. | | `ostra-ratelimit-requests-remaining` | integer | No | Requests left in the current minute. | | `ostra-ratelimit-requests-reset` | integer | No | Seconds until the current minute ends and the count resets. | | `ostra-balance-remaining-usd` | string | No | Workspace credit balance after this request, in USD. Approximate under concurrent requests. | ### Errors | Status | Type | When | |---|---|---| | 400 | `invalid_request` | The body isn’t valid JSON or fails validation; `param` names the field. | | 413 | `payload_too_large` | The request body is larger than the route allows. | | 401 | `authentication_error` | The API key is missing, malformed, unknown, revoked or expired. | | 403 | `permission_error` | The caller’s IP isn’t in the key’s allowlist, or a read-only key called a route that needs full access. | | 402 | `insufficient_balance` | Your credit balance doesn’t cover the request. Nothing is charged. | | 402 | `spend_limit_reached` | The workspace’s monthly spend limit (`scope: "org"`) or this key’s monthly spend cap (`scope: "key"`) is reached. | | 429 | `rate_limit_exceeded` | The workspace’s requests-per-minute limit is reached. Wait `Retry-After` seconds. | | 502 | `upstream_error` | Ostra Decision failed to answer. Nothing was charged. | | 503 | `service_unavailable` | Too many requests are updating the workspace balance at once. Nothing was charged; retry after `Retry-After` (1 s). | | 504 | `upstream_timeout` | Ostra Decision didn’t answer within 10 seconds. Nothing was charged. | ### Example: Failure triage ```bash curl https://ostra.testrelic.ai/api/v1/predict \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "ostra-lite", "task": "triage", "input": { "test_id": "checkout.spec.ts > applies coupon", "run_id": "gha-18241", "context": { "framework": "playwright", "capture_level": "full", "error_message": "TimeoutError: locator.click: Timeout 30000ms exceeded.\n waiting for locator('\''#apply-coupon'\'')", "action": "click", "locator": { "strategy": "css" }, "diff": { "files_changed": 3, "lines_added": 41, "lines_removed": 12, "touches_test_file": false, "touches_selector_file": false, "locator_values_removed": false }, "dom": { "locator_found": true }, "memory": { "runs_50": 40, "flake_rate_50": 0.15, "failed_count_50": 6, "signature_repeat_count": 5 } } } }' ``` ### Example: Test impact ```bash curl https://ostra.testrelic.ai/api/v1/predict \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "ostra-lite", "task": "test_impact", "input": { "diff": { "files_changed": 2, "lines_added": 87, "lines_removed": 14, "touches_test_file": false, "touches_selector_file": false }, "candidates": [ { "test_id": "checkout.spec.ts > applies coupon", "co_change_rate": 0.62, "historical_fail_rate": 0.08 }, { "test_id": "checkout.spec.ts > rejects expired coupon", "co_change_rate": 0.58, "historical_fail_rate": 0.03 }, { "test_id": "search.spec.ts > filters by brand", "co_change_rate": 0.02, "historical_fail_rate": 0.04 } ] } }' ``` ### Example: Selector survival ```bash curl https://ostra.testrelic.ai/api/v1/predict \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "ostra-lite", "task": "selector_survival", "input": { "locators": [ { "locator_id": "checkout.apply-coupon", "strategy": "css", "dom_locator_found": true, "history": [ { "release": "2026.10.1", "resolve_rate": 1 }, { "release": "2026.10.2", "resolve_rate": 0.92 }, { "release": "2026.11.1", "resolve_rate": 0.71 } ] }, { "locator_id": "nav.account-menu", "strategy": "role", "dom_locator_found": true, "history": [ { "release": "2026.10.2", "resolve_rate": 1 }, { "release": "2026.11.1", "resolve_rate": 1 } ] } ] } }' ``` ### Example response (200 · Failure triage) ```json { "id": "req_01JDXK4M7Q2N8R5T6V9W3Y1Z0A", "model": "ostra-lite@v.beta.5", "task": "triage", "cached": false, "billed": { "units": 1, "costNano": 500, "costUsd": "0.0000005" }, "output": { "decision_id": "dec_5b0c9e2a7f3d4e18a6b2c9d0e1f4a7b3", "task": "triage", "decision": "flaky", "value": "flaky", "p": 0.83, "probabilities": { "flaky": 0.83, "product_bug": 0.06, "test_bug": 0.05, "environment": 0.04, "selector_drift": 0.02 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.4.0", "content_hash": "sha256:3f9a1c07d2e4b8a6", "gate_passed": true }, "test_id": "checkout.spec.ts > applies coupon", "run_id": "gha-18241", "error_class": "timeout", "error_class_source": "error_message" } } ``` Reference page: https://docs.testrelic.ai/lores/api/decision/predict ## Run a batch `POST https://ostra.testrelic.ai/api/v1/batch` Runs up to 100 Ostra Decision tasks in one request, mixed tasks allowed, at half the price. Each item is answered independently: one bad item fails on its own and the rest still answer. Results come back in input order, with your `custom_id`. Items run 8 at a time with a 25-second deadline for the whole batch; items that don’t start in time fail with `upstream_timeout` and are refunded. - **Auth:** `Authorization: Bearer `; Full-access key. - **Rate limited:** yes (counts against the workspace’s requests per minute). - **Cost:** Each item costs half its `/predict` price (rounded up to the nano-dollar). Failed and cached items are free. If the balance can’t cover the whole batch, the whole batch is refused with 402. ### Request body | Field | Type | Required | Description | |---|---|---|---| | `model` | string | Yes | Always `ostra-lite`. One of `ostra-lite`. | | `items` | object[] | Yes | The tasks to run. Each item counts as one request against your rate limit. 1–100 items. | | `items[].custom_id` | string | No | Your id for the item, echoed in its result. Longer values are cut to 128 characters. 0–128 characters. | | `items[].task` | string | Yes | Which task to run. One of `triage`, `test_impact`, `selector_survival`. | | `items[].input` | object | Yes | The task’s input, as for `/predict`. | ### Response | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Request id. | | `model` | string | No | The model and release that answered. | | `billed` | object | No | What this request cost. Cached answers cost 0. | | `billed.units` | integer | No | Billable units: 1 per failure (triage), 1 per request (test impact), 1 per locator (selector survival). 0 for a cached answer. | | `billed.costNano` | integer | No | Cost in nano-dollars (1 USD = 1,000,000,000). | | `billed.costUsd` | string | No | Cost in USD as an exact decimal string, e.g. `"0.0000005"`. | | `results` | object[] | No | One result per item, in input order. | | `results[].custom_id` | string | No | Your `custom_id`, when you sent one. | | `results[].ok` | boolean | No | `true` when the item answered. | | `results[].task` | string | No | The item’s task. | | `results[].cached` | boolean | No | When `ok`: the answer came from cache. | | `results[].output` | object | No | When `ok`: the task’s answer, as for `/predict`. | | `results[].error` | object | No | When not `ok`: `{ "type", "message" }`. | | `results[].error.type` | string | No | Error type, e.g. `invalid_request` or `upstream_error`. | | `results[].error.message` | string | No | What went wrong. | ### Response headers | Field | Type | Required | Description | |---|---|---|---| | `x-request-id` | string | No | Id of this request (`req_` + 26 characters). Quote it to support. | | `ostra-ratelimit-requests-limit` | integer | No | Your workspace’s requests-per-minute limit. | | `ostra-ratelimit-requests-remaining` | integer | No | Requests left in the current minute. | | `ostra-ratelimit-requests-reset` | integer | No | Seconds until the current minute ends and the count resets. | | `ostra-balance-remaining-usd` | string | No | Workspace credit balance after this request, in USD. Approximate under concurrent requests. | ### Errors | Status | Type | When | |---|---|---| | 400 | `invalid_request` | `items` is missing, empty, or longer than 100. Per-item problems come back inside `results` instead. | | 413 | `payload_too_large` | The body is larger than 2 MB. | | 401 | `authentication_error` | The API key is missing, malformed, unknown, revoked or expired. | | 403 | `permission_error` | The caller’s IP isn’t in the key’s allowlist, or a read-only key called a route that needs full access. | | 402 | `insufficient_balance` | Your credit balance doesn’t cover the request. Nothing is charged. | | 402 | `spend_limit_reached` | The workspace’s monthly spend limit (`scope: "org"`) or this key’s monthly spend cap (`scope: "key"`) is reached. | | 429 | `rate_limit_exceeded` | The workspace’s requests-per-minute limit is reached. Wait `Retry-After` seconds. | | 503 | `service_unavailable` | Too many requests are updating the workspace balance at once. Nothing was charged; retry after `Retry-After` (1 s). | ### Example: Two triage items and a selector check ```bash curl https://ostra.testrelic.ai/api/v1/batch \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "ostra-lite", "items": [ { "custom_id": "checkout-applies-coupon", "task": "triage", "input": { "test_id": "checkout.spec.ts > applies coupon", "run_id": "gha-18241", "context": { "framework": "playwright", "capture_level": "full", "error_message": "TimeoutError: locator.click: Timeout 30000ms exceeded.\n waiting for locator('\''#apply-coupon'\'')", "action": "click", "locator": { "strategy": "css" }, "diff": { "files_changed": 3, "lines_added": 41, "lines_removed": 12, "touches_test_file": false, "touches_selector_file": false, "locator_values_removed": false }, "dom": { "locator_found": true }, "memory": { "runs_50": 40, "flake_rate_50": 0.15, "failed_count_50": 6, "signature_repeat_count": 5 } } } }, { "custom_id": "orders-lists-orders", "task": "triage", "input": { "test_id": "orders.api.spec.ts > lists orders", "context": { "framework": "jest", "error_message": "Error: Request failed with status code 503 (Service Unavailable) GET /api/orders", "action": "api_call" } } }, { "custom_id": "coupon-locators", "task": "selector_survival", "input": { "locators": [ { "locator_id": "checkout.apply-coupon", "strategy": "css", "dom_locator_found": true, "history": [ { "release": "2026.10.1", "resolve_rate": 1 }, { "release": "2026.10.2", "resolve_rate": 0.92 }, { "release": "2026.11.1", "resolve_rate": 0.71 } ] }, { "locator_id": "nav.account-menu", "strategy": "role", "dom_locator_found": true, "history": [ { "release": "2026.10.2", "resolve_rate": 1 }, { "release": "2026.11.1", "resolve_rate": 1 } ] } ] } } ] }' ``` ### Example response (200 · Results) ```json { "id": "req_01JDXK7Z8Y7X6W5V4U3T2S1R0Q", "model": "ostra-lite@v.beta.5", "billed": { "units": 4, "costNano": 750, "costUsd": "0.00000075" }, "results": [ { "custom_id": "checkout-applies-coupon", "ok": true, "task": "triage", "cached": false, "output": { "decision_id": "dec_5b0c9e2a7f3d4e18a6b2c9d0e1f4a7b3", "task": "triage", "decision": "flaky", "value": "flaky", "p": 0.83, "probabilities": { "flaky": 0.83, "product_bug": 0.06, "test_bug": 0.05, "environment": 0.04, "selector_drift": 0.02 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.4.0", "content_hash": "sha256:3f9a1c07d2e4b8a6", "gate_passed": true }, "test_id": "checkout.spec.ts > applies coupon", "run_id": "gha-18241", "error_class": "timeout", "error_class_source": "error_message" } }, { "custom_id": "orders-lists-orders", "ok": true, "task": "triage", "cached": false, "output": { "decision_id": "dec_6f708192a3b4c5d6e7f8091a2b3c4d5e", "task": "triage", "decision": "environment", "value": "environment", "p": 0.74, "probabilities": { "flaky": 0.12, "product_bug": 0.08, "test_bug": 0.03, "environment": 0.74, "selector_drift": 0.03 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.4.0", "content_hash": "sha256:3f9a1c07d2e4b8a6", "gate_passed": true }, "test_id": "orders.api.spec.ts > lists orders", "run_id": null, "error_class": "network_error", "error_class_source": "error_message" } }, { "custom_id": "coupon-locators", "ok": true, "task": "selector_survival", "cached": false, "output": { "decisions": [ { "decision_id": "dec_4d5e6f708192a3b4c5d6e7f8091a2b3c", "task": "selector_survival", "decision": "breaks", "value": "breaks", "p": 0.78, "probabilities": { "survives": 0.22, "breaks": 0.78 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.2.0", "content_hash": "sha256:5d21f0e9c4a7b3e8", "gate_passed": true }, "locator_id": "checkout.apply-coupon" }, { "decision_id": "dec_5e6f708192a3b4c5d6e7f8091a2b3c4d", "task": "selector_survival", "decision": "survives", "value": "survives", "p": 0.96, "probabilities": { "survives": 0.96, "breaks": 0.04 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.2.0", "content_hash": "sha256:5d21f0e9c4a7b3e8", "gate_passed": true }, "locator_id": "nav.account-menu" } ] } } ] } ``` Reference page: https://docs.testrelic.ai/lores/api/decision/batch ## Create a chat completion `POST https://ostra.testrelic.ai/api/v1/chat/completions` Sends a conversation to Ostra Pro and returns its reply. OpenAI Chat Completions compatible, streamed or not. Any OpenAI client works: set its base URL to `https://ostra.testrelic.ai/api/v1` and its API key to your Ostra key. Thinking is on by default; set its depth with `reasoning_effort`, or turn it off with `chat_template_kwargs.enable_thinking: false`. Fields not listed below (for example `top_p`, `stop`, `seed`, `response_format`, `tool_choice`) are passed to the model unchanged. - **Auth:** `Authorization: Bearer ` or `x-api-key: `; Full-access key. - **Rate limited:** yes (counts against the workspace’s requests per minute). - **Cost:** Input $0.64, cached input $0.06, output $4.47 per 1M tokens; reasoning tokens bill as output. The most a request can cost is held from your balance up front and the rest is refunded when it ends. Failed requests and cold-start timeouts cost nothing; a stream that breaks bills only the tokens produced. ### Request headers | Field | Type | Required | Description | |---|---|---|---| | `x-ostra-session` | string | No | Optional. One value per conversation or agent run (1–128 letters, digits, `_`, `.`, `:` or `-`). Requests with the same value are routed to the same model replica when possible, so more of the conversation’s earlier turns are served from the prompt cache and bill as cached input. Best effort: an unknown value or a different replica just means no cache hit. See [Sessions and caching](https://docs.testrelic.ai/lores/api/sessions-and-caching). | ### Request body | Field | Type | Required | Description | |---|---|---|---| | `model` | string | Yes | Always `ostra-pro`. One of `ostra-pro`. | | `messages` | object[] | Yes | The conversation so far. Up to 8 images and 1 video per request. 1–4000 items. | | `messages[].role` | string | Yes | Who wrote the message. One of `system`, `developer`, `user`, `assistant`, `tool`. | | `messages[].content` | string \| array of parts \| null | No | Text, or an array of parts: `{ "type": "text", "text" }`, `{ "type": "image_url", "image_url": { "url" } }` (https or `data:image/...;base64,` URL) or `{ "type": "video_url", "video_url": { "url" } }`. | | `messages[].tool_calls` | object[] | No | On `assistant` messages: the tool calls the model made, sent back as you received them. | | `messages[].tool_call_id` | string | No | On `tool` messages: the id of the call this message answers. | | `messages[].name` | string | No | Optional participant name. | | `max_completion_tokens` | integer | No | Most output tokens to generate, reasoning included. Values above 32,768 are lowered to 32,768. Takes precedence over `max_tokens`. Range 1–…. Default `8192`. | | `max_tokens` | integer | No | Same as `max_completion_tokens` (older name). Range 1–…. Default `8192`. | | `stream` | boolean | No | Stream the reply as server-sent events. See [Streaming](https://docs.testrelic.ai/lores/api/streaming). Default `false`. | | `stream_options` | object | No | With `stream`. Set `include_usage: true` to get a final chunk with token usage. | | `stream_options.include_usage` | boolean | No | Send a usage-only chunk before `[DONE]`. | | `reasoning_effort` | string | No | How long the model thinks before answering. Lower is faster and cheaper. One of `low`, `medium`, `xhigh`. Default `"xhigh"`. | | `chat_template_kwargs` | object | No | Model template switches. | | `chat_template_kwargs.enable_thinking` | boolean | No | `false` answers directly, with no reasoning. Default `true`. | | `tools` | object[] | No | Functions the model may call, in OpenAI format. 0–128 items. | | `tools[].type` | string | Yes | Always `function`. One of `function`. | | `tools[].function` | object | Yes | `{ "name", "description", "parameters" }`, where `parameters` is a JSON Schema. | | `tool_choice` | string \| object | No | `auto` (default), `none`, `required`, or a specific function, as in OpenAI. | | `temperature` | number | No | Sampling temperature. Range 0–2. | | `n` | integer | No | Only `1` is supported. One of `1`. | ### Response | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Completion id. | | `object` | string | No | `chat.completion` (or `chat.completion.chunk` when streaming). | | `created` | integer | No | Unix time. | | `model` | string | No | `ostra-pro`. | | `choices` | object[] | No | One choice. | | `choices[].index` | integer | No | Always 0. | | `choices[].message` | object | No | The reply. | | `choices[].message.role` | string | No | `assistant`. | | `choices[].message.content` | string \| null | No | The answer text. | | `choices[].message.reasoning_content` | string \| null | No | The model’s reasoning, when thinking is on. | | `choices[].message.tool_calls` | object[] | No | Tool calls the model wants you to run: `{ "id", "type": "function", "function": { "name", "arguments" } }`. | | `choices[].finish_reason` | string | No | `stop`, `length` (hit the token limit) or `tool_calls`. | | `usage` | object | No | Token counts this request was billed for. | | `usage.prompt_tokens` | integer | No | Input tokens, cached ones included. | | `usage.completion_tokens` | integer | No | Output tokens, reasoning included. | | `usage.total_tokens` | integer | No | Sum of the two. | | `usage.prompt_tokens_details` | object | No | Breakdown of input. | | `usage.prompt_tokens_details.cached_tokens` | integer | No | Input tokens served from the prompt cache, billed at the cached rate. | ### Response headers | Field | Type | Required | Description | |---|---|---|---| | `x-request-id` | string | No | Id of this request (`req_` + 26 characters). Quote it to support. | | `ostra-ratelimit-requests-limit` | integer | No | Your workspace’s requests-per-minute limit. | | `ostra-ratelimit-requests-remaining` | integer | No | Requests left in the current minute. | | `ostra-ratelimit-requests-reset` | integer | No | Seconds until the current minute ends and the count resets. | | `ostra-model-version` | string | No | Ostra Pro version that answered: `1.0`. | | `ostra-cost-usd` | string | No | Non-streaming only: what this request cost. | | `ostra-balance-remaining-usd` | string | No | Workspace credit balance after this request, in USD. Approximate under concurrent requests. | ### Errors | Status | Type | When | |---|---|---| | 400 | `invalid_request` | Validation failed (`param` names the field), or the model rejected the request. | | 413 | `payload_too_large` | The body is larger than 5.5 MB (images and video count). | | 401 | `authentication_error` | The API key is missing, malformed, unknown, revoked or expired. | | 403 | `permission_error` | The caller’s IP isn’t in the key’s allowlist, or a read-only key called a route that needs full access. | | 402 | `insufficient_balance` | Your balance doesn’t cover the request’s maximum cost. Top up or lower `max_tokens`. | | 402 | `spend_limit_reached` | The workspace’s monthly spend limit (`scope: "org"`) or this key’s monthly spend cap (`scope: "key"`) is reached. | | 429 | `rate_limit_exceeded` | The workspace’s requests-per-minute limit is reached. Wait `Retry-After` seconds. | | 502 | `upstream_error` | Ostra Pro failed to answer. Nothing was charged. | | 503 | `service_unavailable` | Ostra Pro is starting after a quiet period (`state: "starting"`, retry after 60 s) or at capacity (retry after 5 s). Nothing was charged. | ### Example: Ask a question ```bash curl https://ostra.testrelic.ai/api/v1/chat/completions \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'x-ostra-session: checkout-suite-run-42' \ -d '{ "model": "ostra-pro", "messages": [ { "role": "system", "content": "You are a senior test engineer. Answer briefly." }, { "role": "user", "content": "This Playwright test times out on CI but passes locally. Why, and how do I fix it?" } ], "reasoning_effort": "low", "max_tokens": 1024 }' ``` ### Example: Stream the reply ```bash curl -N https://ostra.testrelic.ai/api/v1/chat/completions \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'x-ostra-session: checkout-suite-run-42' \ -d '{ "model": "ostra-pro", "messages": [ { "role": "system", "content": "You are a senior test engineer. Answer briefly." }, { "role": "user", "content": "This Playwright test times out on CI but passes locally. Why, and how do I fix it?" } ], "reasoning_effort": "low", "max_tokens": 1024, "stream": true, "stream_options": { "include_usage": true } }' ``` ### Example: Call a tool ```bash curl https://ostra.testrelic.ai/api/v1/chat/completions \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "ostra-pro", "messages": [ { "role": "user", "content": "Run tests/checkout.spec.ts and tell me what failed." } ], "tools": [ { "type": "function", "function": { "name": "run_tests", "description": "Run part of the test suite and return its output.", "parameters": { "type": "object", "properties": { "path": { "type": "string", "description": "Test file or directory" } }, "required": [ "path" ] } } } ], "reasoning_effort": "low" }' ``` ### Example: Send a screenshot ```bash curl https://ostra.testrelic.ai/api/v1/chat/completions \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model": "ostra-pro", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "List the visual bugs on this checkout page." }, { "type": "image_url", "image_url": { "url": "https://docs.testrelic.ai/img/samples/checkout.png" } } ] } ], "reasoning_effort": "medium" }' ``` ### Example response (200 · Completion) ```json { "id": "chatcmpl-7f3a9c1e2b4d", "object": "chat.completion", "created": 1795615200, "model": "ostra-pro", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "CI machines are slower and run tests in parallel, so the coupon button renders after your 30 s timeout. Wait for the state you need instead of a fixed time: `await expect(page.getByRole('button', { name: 'Apply coupon' })).toBeEnabled()` before clicking, and stop sharing the cart between tests so parallel workers don’t race.", "reasoning_content": "Timeout only on CI: likely slower runners or shared state between parallel tests…" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 52, "completion_tokens": 170, "total_tokens": 222, "prompt_tokens_details": { "cached_tokens": 0 } } } ``` Reference page: https://docs.testrelic.ai/lores/api/pro/chat-completions ## Create a message `POST https://ostra.testrelic.ai/api/v1/messages` Sends a conversation to Ostra Pro in Anthropic Messages format. Anthropic SDKs and Anthropic-compatible tools work unchanged. Point an Anthropic client at base URL `https://ostra.testrelic.ai/api` (the SDK adds `/v1/messages`) with your Ostra key. The `model` you send is ignored: Ostra Pro always answers, and the response says `ostra-pro`. Some Anthropic options are adapted: `output_config.effort` above `medium` runs as `medium`; `tool_choice` `any` or `tool` runs as `auto`, and `none` removes the tools; mid-conversation `system` messages are folded into the next user turn. Errors use the Anthropic error shape. `count_tokens` isn’t supported. - **Auth:** `x-api-key: ` or `Authorization: Bearer `; Full-access key. - **Rate limited:** yes (counts against the workspace’s requests per minute). - **Cost:** Same rates as chat completions. Input is `input_tokens + cache_read_input_tokens + cache_creation_input_tokens`; `cache_read_input_tokens` bill at the cached rate. ### Request headers | Field | Type | Required | Description | |---|---|---|---| | `x-ostra-session` | string | No | Optional. One value per conversation or agent run (1–128 letters, digits, `_`, `.`, `:` or `-`). Requests with the same value are routed to the same model replica when possible, so more of the conversation’s earlier turns are served from the prompt cache and bill as cached input. Best effort: an unknown value or a different replica just means no cache hit. See [Sessions and caching](https://docs.testrelic.ai/lores/api/sessions-and-caching). | | `anthropic-version` | string | No | Accepted and ignored, so SDK defaults work. | | `anthropic-beta` | string | No | Accepted and ignored. | ### Request body | Field | Type | Required | Description | |---|---|---|---| | `model` | string | No | Optional and ignored: Ostra Pro answers whatever you send. Anthropic SDKs require one; use `ostra-pro`. | | `messages` | object[] | Yes | The conversation. Up to 8 `image` blocks per request, including those inside `tool_result`. 1–4000 items. | | `messages[].role` | string | Yes | Who wrote it. `system` mid-conversation is folded into the next user turn. One of `user`, `assistant`, `system`. | | `messages[].content` | string \| array of blocks | Yes | Text, or content blocks (`text`, `image`, `tool_use`, `tool_result`, `thinking`). | | `system` | string \| array of text blocks | No | System prompt. | | `max_tokens` | integer | No | Most output tokens, thinking included. Above 32,768 is lowered to 32,768. Range 1–…. Default `8192`. | | `stream` | boolean | No | Stream Anthropic events (`message_start`, `content_block_*`, `message_delta`, `message_stop`). Default `false`. | | `tools` | object[] | No | Tools in Anthropic format: `{ "name", "description", "input_schema" }`. 0–128 items. | | `tool_choice` | object | No | `{ "type": "auto" }` or `{ "type": "none" }`. `any` and `tool` run as `auto`. | | `thinking` | object | No | Passed through, e.g. `{ "type": "enabled", "budget_tokens": 2048 }`. | | `output_config` | object | No | `effort`: `low` or `medium` (higher values run as `medium`). | | `temperature` | number | No | Sampling temperature. Range 0–1. | | `metadata` | object | No | Passed through. | ### Response | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Message id. | | `type` | string | No | `message`. | | `role` | string | No | `assistant`. | | `model` | string | No | `ostra-pro`. | | `content` | object[] | No | Content blocks: `thinking` (when thinking is on), `text`, `tool_use`. | | `stop_reason` | string | No | `end_turn`, `max_tokens` or `tool_use`. | | `stop_sequence` | string \| null | No | The stop sequence that ended the answer, if any. | | `usage` | object | No | `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`. | ### Response headers | Field | Type | Required | Description | |---|---|---|---| | `x-request-id` | string | No | Id of this request (`req_` + 26 characters). Quote it to support. | | `ostra-ratelimit-requests-limit` | integer | No | Your workspace’s requests-per-minute limit. | | `ostra-ratelimit-requests-remaining` | integer | No | Requests left in the current minute. | | `ostra-ratelimit-requests-reset` | integer | No | Seconds until the current minute ends and the count resets. | | `request-id` | string | No | Same as `x-request-id`, under Anthropic’s name. | | `ostra-model-version` | string | No | `1.0`. | | `ostra-cost-usd` | string | No | Non-streaming only: what this request cost. | | `ostra-balance-remaining-usd` | string | No | Workspace credit balance after this request, in USD. Approximate under concurrent requests. | ### Errors | Status | Type | When | |---|---|---| | 400 | `invalid_request_error` | Validation failed, or the model rejected the request. | | 401 | `authentication_error` | Missing or invalid key. | | 402 | `billing_error` | Balance too low, or a spend limit reached (`ostra_type` says which). | | 403 | `permission_error` | IP not allowed, or a read-only key. | | 413 | `request_too_large` | Body larger than 5.5 MB. | | 429 | `rate_limit_error` | Rate limit reached. | | 502 | `api_error` | Ostra Pro failed to answer. | | 503 | `overloaded_error` | Ostra Pro is starting or at capacity. Retry after `Retry-After`. | ### Example: Ask a question ```bash curl https://ostra.testrelic.ai/api/v1/messages \ -H "x-api-key: $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'anthropic-version: 2023-06-01' \ -H 'x-ostra-session: checkout-suite-run-42' \ -d '{ "model": "ostra-pro", "max_tokens": 1024, "system": "You are a senior test engineer. Answer briefly.", "messages": [ { "role": "user", "content": "This Playwright test times out on CI but passes locally. Why, and how do I fix it?" } ] }' ``` ### Example response (200 · Message) ```json { "id": "msg_01JDXKA2B3C4D5E6F7G8H9J0K1", "type": "message", "role": "assistant", "model": "ostra-pro", "content": [ { "type": "thinking", "thinking": "Timeout only on CI: likely slower runners or shared state between parallel tests…", "signature": "" }, { "type": "text", "text": "CI machines are slower and run tests in parallel, so the coupon button renders after your 30 s timeout. Wait for the state you need instead of a fixed time: `await expect(page.getByRole('button', { name: 'Apply coupon' })).toBeEnabled()` before clicking, and stop sharing the cart between tests so parallel workers don’t race." } ], "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 52, "output_tokens": 170, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0 } } ``` Reference page: https://docs.testrelic.ai/lores/api/pro/messages ## Warm up Ostra Pro `POST https://ostra.testrelic.ai/api/v1/warm` Starts Ostra Pro if it’s asleep, so the next request doesn’t wait for a cold start. Ostra Pro scales to zero after 5 quiet minutes. A cold start usually takes under a minute, and up to about 5 minutes the first time after a model update. Call this before a burst of work (a CI job, an agent run) and poll it until it says `ready`. - **Auth:** `Authorization: Bearer ` or `x-api-key: `; any key. - **Rate limited:** no. - **Cost:** Free. Any key works, read-only included. Not rate limited. ### Response | Field | Type | Required | Description | |---|---|---|---| | `model` | string | No | `ostra-pro`. | | `state` | string | No | `ready`: a replica is up. `starting`: one is starting; poll again. One of `ready`, `starting`. | | `retry_after_seconds` | integer | No | With `starting`: when to poll again (30). | ### Response headers | Field | Type | Required | Description | |---|---|---|---| | `x-request-id` | string | No | Id of this request (`req_` + 26 characters). Quote it to support. | ### Errors | Status | Type | When | |---|---|---| | 401 | `authentication_error` | The API key is missing, malformed, unknown, revoked or expired. | | 403 | `permission_error` | The caller’s IP isn’t in the key’s allowlist. | | 502 | `upstream_error` | Ostra Pro answered unexpectedly. Retry shortly. | ### Example: Warm up ```bash curl -X POST https://ostra.testrelic.ai/api/v1/warm \ -H "Authorization: Bearer $OSTRA_API_KEY" ``` ### Example response (200 · Ready) ```json { "model": "ostra-pro", "state": "ready" } ``` Reference page: https://docs.testrelic.ai/lores/api/pro/warm # MCP tools ## MCP tool `ostra_triage_failure` Ostra Decision: Failure triage. Classifies a failing test as flaky, product bug, test bug, environment or selector drift, with a confidence. - **Access:** Full-access key. - **Cost:** $0.50 per 1M failures, same as REST. Cached answers are free. ### Arguments | Field | Type | Required | Description | |---|---|---|---| | `test_id` | string | No | Your identifier for the failing test, echoed back in the answer. Use a stable name, e.g. `checkout.spec.ts > applies coupon`. 1–512 characters. | | `run_id` | string | No | Your identifier for the CI run, echoed back. 1–512 characters. | | `context` | object | Yes | Evidence about the failure. Every field is optional except that `error_class` or `error_message` must be present. Missing evidence is treated as missing, never as zero, so send what you have. | | `context.id` | string | No | Your own id for this failure record. 1–512 characters. | | `context.framework` | string | No | Test framework that produced the failure. One of `playwright`, `cypress`, `jest`, `selenium`. | | `context.capture_level` | string | No | How much of the failure was captured (trace, DOM, network): `full`, `partial` or `none`. One of `full`, `partial`, `none`. | | `context.error_class` | string | No | Normalized class of the error. Required unless you send `error_message`. One of `timeout`, `locator_not_found`, `stale_element`, `strict_mode_violation`, `assertion_mismatch`, `network_error`, `script_error`, `unmapped`. | | `context.error_message` | string | No | The raw error text. Used only to derive `error_class` when you don’t send one; the answer reports which you used in `error_class_source`. 0–8000 characters. | | `context.action` | string | No | The step that failed. One of `click`, `fill`, `navigate`, `assert`, `wait_for`, `api_call`, `screenshot`. | | `context.locator` | object | No | The locator the failing step used. | | `context.locator.strategy` | string | No | How the element was located. One of `css`, `xpath`, `test_id`, `text`, `role`, `none`. | | `context.diff` | object | No | The code change the run tested. | | `context.diff.files_changed` | integer | No | Files changed in the commit or pull request under test. Range 0–1000000. | | `context.diff.lines_added` | integer | No | Lines added. Range 0–1000000. | | `context.diff.lines_removed` | integer | No | Lines removed. Range 0–1000000. | | `context.diff.touches_test_file` | boolean | No | The change edits a test file. | | `context.diff.touches_selector_file` | boolean | No | The change edits a page object, locator map or other file that defines selectors. | | `context.diff.locator_values_removed` | boolean | No | The change removes or renames a locator value (an id, test id, class or text) that the test uses. | | `context.dom` | object | No | What the page looked like at failure time. | | `context.dom.locator_found` | boolean | No | The locator still matched an element in the captured DOM. | | `context.memory` | object | No | This test’s recent history, over its last 50 runs at most. | | `context.memory.runs_50` | integer | No | Runs in the window. Range 0–50. | | `context.memory.failed_count_50` | integer | No | Failed runs in the window. Range 0–50. | | `context.memory.passed_on_retry_50` | integer | No | Runs that failed and then passed on retry. Range 0–50. | | `context.memory.consecutive_failures_50` | integer | No | Failures in a row up to this one. Range 0–50. | | `context.memory.flake_rate_50` | number | No | Share of runs in the window that were flaky, 0–1. Range 0–1. | | `context.memory.signature_repeat_count` | integer | No | How many times this exact error signature has been seen before. Range 0–1000000. | ### Result (`structuredContent`) | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Request id. | | `model` | string | No | Model and release, e.g. `ostra-lite@v.beta.5`. | | `task` | string | No | The task. | | `cached` | boolean | No | Answered from the 24-hour cache. | | `billed` | object | No | What this request cost. Cached answers cost 0. | | `billed.units` | integer | No | Billable units: 1 per failure (triage), 1 per request (test impact), 1 per locator (selector survival). 0 for a cached answer. | | `billed.costNano` | integer | No | Cost in nano-dollars (1 USD = 1,000,000,000). | | `billed.costUsd` | string | No | Cost in USD as an exact decimal string, e.g. `"0.0000005"`. | | `output` | object | No | The task’s answer, the same shape as `/predict`. | ### Example ```bash curl https://ostra.testrelic.ai/mcp \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'Accept: application/json, text/event-stream' \ -H 'MCP-Protocol-Version: 2025-06-18' \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "ostra_triage_failure", "arguments": { "test_id": "checkout.spec.ts > applies coupon", "run_id": "gha-18241", "context": { "framework": "playwright", "capture_level": "full", "error_message": "TimeoutError: locator.click: Timeout 30000ms exceeded.\n waiting for locator('\''#apply-coupon'\'')", "action": "click", "locator": { "strategy": "css" }, "diff": { "files_changed": 3, "lines_added": 41, "lines_removed": 12, "touches_test_file": false, "touches_selector_file": false, "locator_values_removed": false }, "dom": { "locator_found": true }, "memory": { "runs_50": 40, "flake_rate_50": 0.15, "failed_count_50": 6, "signature_repeat_count": 5 } } } } }' ``` ```json { "content": [ { "type": "text", "text": "{\"id\":\"req_01JDXK4M7Q2N8R5T6V9W3Y1Z0A\",\"model\":\"ostra-lite@v.beta.5\",\"task\":\"triage\",\"cached\":false,\"billed\":{\"units\":1,\"costNano\":500,\"costUsd\":\"0.0000005\"},\"output\":{\"decision_id\":\"dec_5b0c9e2a7f3d4e18a6b2c9d0e1f4a7b3\",\"task\":\"triage\",\"decision\":\"flaky\",\"value\":\"flaky\",\"p\":0.83,\"probabilities\":{\"flaky\":0.83,\"product_bug\":0.06,\"test_bug\":0.05,\"environment\":0.04,\"selector_drift\":0.02},\"threshold\":0.6,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.4.0\",\"content_hash\":\"sha256:3f9a1c07d2e4b8a6\",\"gate_passed\":true},\"test_id\":\"checkout.spec.ts > applies coupon\",\"run_id\":\"gha-18241\",\"error_class\":\"timeout\",\"error_class_source\":\"error_message\"}}" } ], "structuredContent": { "id": "req_01JDXK4M7Q2N8R5T6V9W3Y1Z0A", "model": "ostra-lite@v.beta.5", "task": "triage", "cached": false, "billed": { "units": 1, "costNano": 500, "costUsd": "0.0000005" }, "output": { "decision_id": "dec_5b0c9e2a7f3d4e18a6b2c9d0e1f4a7b3", "task": "triage", "decision": "flaky", "value": "flaky", "p": 0.83, "probabilities": { "flaky": 0.83, "product_bug": 0.06, "test_bug": 0.05, "environment": 0.04, "selector_drift": 0.02 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.4.0", "content_hash": "sha256:3f9a1c07d2e4b8a6", "gate_passed": true }, "test_id": "checkout.spec.ts > applies coupon", "run_id": "gha-18241", "error_class": "timeout", "error_class_source": "error_message" } }, "isError": false } ``` Reference page: https://docs.testrelic.ai/lores/api/mcp/ostra-triage-failure ## MCP tool `ostra_select_impacted_tests` Ostra Decision: Test impact. Given a code change and candidate tests, returns the tests the change is likely to affect, best first. - **Access:** Full-access key. - **Cost:** $2.00 per 1M requests, whatever the number of candidates. ### Arguments | Field | Type | Required | Description | |---|---|---|---| | `diff` | object | Yes | The code change to evaluate. | | `diff.files_changed` | integer | No | Files changed in the commit or pull request under test. Range 0–1000000. | | `diff.lines_added` | integer | No | Lines added. Range 0–1000000. | | `diff.lines_removed` | integer | No | Lines removed. Range 0–1000000. | | `diff.touches_test_file` | boolean | No | The change edits a test file. | | `diff.touches_selector_file` | boolean | No | The change edits a page object, locator map or other file that defines selectors. | | `candidates` | object[] | Yes | The tests that could be affected. One request is billed once, whatever the number of candidates. 1–2000 items. | | `candidates[].test_id` | string | Yes | Your identifier for the test. 1–512 characters. | | `candidates[].co_change_rate` | number | No | How often this test’s files change together with the changed files, 0–1. Range 0–1. | | `candidates[].historical_fail_rate` | number | No | The test’s historical failure rate, 0–1. Range 0–1. | ### Result (`structuredContent`) | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Request id. | | `model` | string | No | Model and release, e.g. `ostra-lite@v.beta.5`. | | `task` | string | No | The task. | | `cached` | boolean | No | Answered from the 24-hour cache. | | `billed` | object | No | What this request cost. Cached answers cost 0. | | `billed.units` | integer | No | Billable units: 1 per failure (triage), 1 per request (test impact), 1 per locator (selector survival). 0 for a cached answer. | | `billed.costNano` | integer | No | Cost in nano-dollars (1 USD = 1,000,000,000). | | `billed.costUsd` | string | No | Cost in USD as an exact decimal string, e.g. `"0.0000005"`. | | `output` | object | No | The task’s answer, the same shape as `/predict`. | ### Example ```bash curl https://ostra.testrelic.ai/mcp \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'Accept: application/json, text/event-stream' \ -H 'MCP-Protocol-Version: 2025-06-18' \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "ostra_select_impacted_tests", "arguments": { "diff": { "files_changed": 2, "lines_added": 87, "lines_removed": 14, "touches_test_file": false, "touches_selector_file": false }, "candidates": [ { "test_id": "checkout.spec.ts > applies coupon", "co_change_rate": 0.62, "historical_fail_rate": 0.08 }, { "test_id": "checkout.spec.ts > rejects expired coupon", "co_change_rate": 0.58, "historical_fail_rate": 0.03 }, { "test_id": "search.spec.ts > filters by brand", "co_change_rate": 0.02, "historical_fail_rate": 0.04 } ] } } }' ``` ```json { "content": [ { "type": "text", "text": "{\"id\":\"req_01JDXK5A1B2C3D4E5F6G7H8J9K\",\"model\":\"ostra-lite@v.beta.5\",\"task\":\"test_impact\",\"cached\":false,\"billed\":{\"units\":1,\"costNano\":2000,\"costUsd\":\"0.000002\"},\"output\":{\"decisions\":[{\"decision_id\":\"dec_1a2b3c4d5e6f708192a3b4c5d6e7f809\",\"task\":\"test_impact\",\"decision\":\"impacted\",\"value\":\"impacted\",\"p\":0.91,\"probabilities\":{\"impacted\":0.91,\"not_impacted\":0.09},\"threshold\":0.55,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.1.2\",\"content_hash\":\"sha256:a07be3315c9d4f12\",\"gate_passed\":true},\"test_id\":\"checkout.spec.ts > applies coupon\"},{\"decision_id\":\"dec_2b3c4d5e6f708192a3b4c5d6e7f8091a\",\"task\":\"test_impact\",\"decision\":\"impacted\",\"value\":\"impacted\",\"p\":0.86,\"probabilities\":{\"impacted\":0.86,\"not_impacted\":0.14},\"threshold\":0.55,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.1.2\",\"content_hash\":\"sha256:a07be3315c9d4f12\",\"gate_passed\":true},\"test_id\":\"checkout.spec.ts > rejects expired coupon\"},{\"decision_id\":\"dec_3c4d5e6f708192a3b4c5d6e7f8091a2b\",\"task\":\"test_impact\",\"decision\":\"not_impacted\",\"value\":\"not_impacted\",\"p\":0.97,\"probabilities\":{\"impacted\":0.03,\"not_impacted\":0.97},\"threshold\":0.55,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.1.2\",\"content_hash\":\"sha256:a07be3315c9d4f12\",\"gate_passed\":true},\"test_id\":\"search.spec.ts > filters by brand\"}],\"impacted\":[\"checkout.spec.ts > applies coupon\",\"checkout.spec.ts > rejects expired coupon\"],\"selection\":\"model\"}}" } ], "structuredContent": { "id": "req_01JDXK5A1B2C3D4E5F6G7H8J9K", "model": "ostra-lite@v.beta.5", "task": "test_impact", "cached": false, "billed": { "units": 1, "costNano": 2000, "costUsd": "0.000002" }, "output": { "decisions": [ { "decision_id": "dec_1a2b3c4d5e6f708192a3b4c5d6e7f809", "task": "test_impact", "decision": "impacted", "value": "impacted", "p": 0.91, "probabilities": { "impacted": 0.91, "not_impacted": 0.09 }, "threshold": 0.55, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.1.2", "content_hash": "sha256:a07be3315c9d4f12", "gate_passed": true }, "test_id": "checkout.spec.ts > applies coupon" }, { "decision_id": "dec_2b3c4d5e6f708192a3b4c5d6e7f8091a", "task": "test_impact", "decision": "impacted", "value": "impacted", "p": 0.86, "probabilities": { "impacted": 0.86, "not_impacted": 0.14 }, "threshold": 0.55, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.1.2", "content_hash": "sha256:a07be3315c9d4f12", "gate_passed": true }, "test_id": "checkout.spec.ts > rejects expired coupon" }, { "decision_id": "dec_3c4d5e6f708192a3b4c5d6e7f8091a2b", "task": "test_impact", "decision": "not_impacted", "value": "not_impacted", "p": 0.97, "probabilities": { "impacted": 0.03, "not_impacted": 0.97 }, "threshold": 0.55, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.1.2", "content_hash": "sha256:a07be3315c9d4f12", "gate_passed": true }, "test_id": "search.spec.ts > filters by brand" } ], "impacted": [ "checkout.spec.ts > applies coupon", "checkout.spec.ts > rejects expired coupon" ], "selection": "model" } }, "isError": false } ``` Reference page: https://docs.testrelic.ai/lores/api/mcp/ostra-select-impacted-tests ## MCP tool `ostra_check_selector` Ostra Decision: Selector survival. Predicts whether each locator will survive the next release or break. - **Access:** Full-access key. - **Cost:** $0.25 per 1M locators. ### Arguments | Field | Type | Required | Description | |---|---|---|---| | `locators` | object[] | Yes | The locators to check. Billed per locator. 1–500 items. | | `locators[].locator_id` | string | No | Your identifier for the locator, echoed back. 1–512 characters. | | `locators[].strategy` | string | No | How the locator finds its element. One of `css`, `xpath`, `test_id`, `text`, `role`, `none`. | | `locators[].dom_locator_found` | boolean | No | The locator matches an element in the current build’s DOM. | | `locators[].history` | object[] | No | How well the locator resolved in past releases, oldest first. 0–100 items. | | `locators[].history[].release` | string | Yes | Release label, e.g. `2026.11.1`. 1–128 characters. | | `locators[].history[].resolve_rate` | number | No | Share of runs in that release where the locator resolved, 0–1. Range 0–1. | ### Result (`structuredContent`) | Field | Type | Required | Description | |---|---|---|---| | `id` | string | No | Request id. | | `model` | string | No | Model and release, e.g. `ostra-lite@v.beta.5`. | | `task` | string | No | The task. | | `cached` | boolean | No | Answered from the 24-hour cache. | | `billed` | object | No | What this request cost. Cached answers cost 0. | | `billed.units` | integer | No | Billable units: 1 per failure (triage), 1 per request (test impact), 1 per locator (selector survival). 0 for a cached answer. | | `billed.costNano` | integer | No | Cost in nano-dollars (1 USD = 1,000,000,000). | | `billed.costUsd` | string | No | Cost in USD as an exact decimal string, e.g. `"0.0000005"`. | | `output` | object | No | The task’s answer, the same shape as `/predict`. | ### Example ```bash curl https://ostra.testrelic.ai/mcp \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'Accept: application/json, text/event-stream' \ -H 'MCP-Protocol-Version: 2025-06-18' \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "ostra_check_selector", "arguments": { "locators": [ { "locator_id": "checkout.apply-coupon", "strategy": "css", "dom_locator_found": true, "history": [ { "release": "2026.10.1", "resolve_rate": 1 }, { "release": "2026.10.2", "resolve_rate": 0.92 }, { "release": "2026.11.1", "resolve_rate": 0.71 } ] }, { "locator_id": "nav.account-menu", "strategy": "role", "dom_locator_found": true, "history": [ { "release": "2026.10.2", "resolve_rate": 1 }, { "release": "2026.11.1", "resolve_rate": 1 } ] } ] } } }' ``` ```json { "content": [ { "type": "text", "text": "{\"id\":\"req_01JDXK6P7Q8R9S0T1U2V3W4X5Y\",\"model\":\"ostra-lite@v.beta.5\",\"task\":\"selector_survival\",\"cached\":false,\"billed\":{\"units\":2,\"costNano\":500,\"costUsd\":\"0.0000005\"},\"output\":{\"decisions\":[{\"decision_id\":\"dec_4d5e6f708192a3b4c5d6e7f8091a2b3c\",\"task\":\"selector_survival\",\"decision\":\"breaks\",\"value\":\"breaks\",\"p\":0.78,\"probabilities\":{\"survives\":0.22,\"breaks\":0.78},\"threshold\":0.6,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.2.0\",\"content_hash\":\"sha256:5d21f0e9c4a7b3e8\",\"gate_passed\":true},\"locator_id\":\"checkout.apply-coupon\"},{\"decision_id\":\"dec_5e6f708192a3b4c5d6e7f8091a2b3c4d\",\"task\":\"selector_survival\",\"decision\":\"survives\",\"value\":\"survives\",\"p\":0.96,\"probabilities\":{\"survives\":0.96,\"breaks\":0.04},\"threshold\":0.6,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.2.0\",\"content_hash\":\"sha256:5d21f0e9c4a7b3e8\",\"gate_passed\":true},\"locator_id\":\"nav.account-menu\"}]}}" } ], "structuredContent": { "id": "req_01JDXK6P7Q8R9S0T1U2V3W4X5Y", "model": "ostra-lite@v.beta.5", "task": "selector_survival", "cached": false, "billed": { "units": 2, "costNano": 500, "costUsd": "0.0000005" }, "output": { "decisions": [ { "decision_id": "dec_4d5e6f708192a3b4c5d6e7f8091a2b3c", "task": "selector_survival", "decision": "breaks", "value": "breaks", "p": 0.78, "probabilities": { "survives": 0.22, "breaks": 0.78 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.2.0", "content_hash": "sha256:5d21f0e9c4a7b3e8", "gate_passed": true }, "locator_id": "checkout.apply-coupon" }, { "decision_id": "dec_5e6f708192a3b4c5d6e7f8091a2b3c4d", "task": "selector_survival", "decision": "survives", "value": "survives", "p": 0.96, "probabilities": { "survives": 0.96, "breaks": 0.04 }, "threshold": 0.6, "defer_reason": null, "decided_by": "model", "rule": null, "model": { "release": "v.beta.5", "version": "1.2.0", "content_hash": "sha256:5d21f0e9c4a7b3e8", "gate_passed": true }, "locator_id": "nav.account-menu" } ] } }, "isError": false } ``` Reference page: https://docs.testrelic.ai/lores/api/mcp/ostra-check-selector ## MCP tool `ostra_pro_chat` Ostra Pro: ask the testing model. Asks Ostra Pro, TestRelic’s testing model, a question: write or fix a test, analyse a failure or log, read a screenshot. Send either `prompt` (with optional `system` and `images`) or a full `messages` array, not both. Long answers can stream progress: see [MCP overview](https://docs.testrelic.ai/lores/api/mcp/overview). - **Access:** Full-access key. - **Cost:** Per token, the same as chat completions. ### Arguments | Field | Type | Required | Description | |---|---|---|---| | `prompt` | string | No | The question or task. Use this or `messages`. | | `system` | string | No | System prompt to go with `prompt`. | | `images` | string[] | No | Image URLs to go with `prompt`: `https://…` or `data:image/…;base64,…`. 0–8 items. | | `messages` | object[] | No | A full OpenAI-style conversation instead of `prompt`/`system`/`images`. | | `max_tokens` | integer | No | Most output tokens, reasoning included. Range 1–32768. Default `8192`. | | `reasoning_effort` | string | No | How long to think; `none` turns thinking off. One of `none`, `low`, `medium`, `xhigh`. Default `"xhigh"`. | | `temperature` | number | No | Sampling temperature. Range 0–2. | | `session` | string | No | Optional conversation id, one per conversation or agent run (like `x-ostra-session`). Calls with the same value go to the same model replica when possible (best effort), so follow-up calls more often bill a repeated prompt prefix as cached input. | ### Result (`structuredContent`) | Field | Type | Required | Description | |---|---|---|---| | `model` | string | No | `ostra-pro`. | | `version` | string | No | `1.0`. | | `content` | string | No | The answer (also the tool result’s text content). | | `reasoning_content` | string | No | The reasoning, when thinking was on. | | `finish_reason` | string | No | `stop`, `length` or `tool_calls`. | | `usage` | object | No | `input_tokens`, `cached_input_tokens`, `output_tokens`. | | `cost_usd` | string | No | What the call cost. | | `balance_usd` | string | No | Balance after the call. | | `request_id` | string | No | Request id. | ### Example ```bash curl https://ostra.testrelic.ai/mcp \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'Accept: application/json, text/event-stream' \ -H 'MCP-Protocol-Version: 2025-06-18' \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "ostra_pro_chat", "arguments": { "prompt": "This Playwright test times out on CI but passes locally. Why, and how do I fix it?", "reasoning_effort": "low", "max_tokens": 2048, "session": "checkout-fix-1" } } }' ``` ```json { "content": [ { "type": "text", "text": "CI machines are slower and run tests in parallel, so the coupon button renders after your 30 s timeout. Wait for the state you need instead of a fixed time: `await expect(page.getByRole('button', { name: 'Apply coupon' })).toBeEnabled()` before clicking, and stop sharing the cart between tests so parallel workers don’t race." } ], "structuredContent": { "model": "ostra-pro", "version": "1.0", "content": "CI machines are slower and run tests in parallel, so the coupon button renders after your 30 s timeout. Wait for the state you need instead of a fixed time: `await expect(page.getByRole('button', { name: 'Apply coupon' })).toBeEnabled()` before clicking, and stop sharing the cart between tests so parallel workers don’t race.", "finish_reason": "stop", "usage": { "input_tokens": 61, "cached_input_tokens": 0, "output_tokens": 170 }, "cost_usd": "0.00079894", "balance_usd": "24.99920106", "request_id": "req_01JDXKB3C4D5E6F7G8H9J0K1L2" }, "isError": false } ``` Reference page: https://docs.testrelic.ai/lores/api/mcp/ostra-pro-chat ## MCP tool `ostra_list_models` List Ostra models. Lists the Ostra models, their tasks and prices. Free, and works with read-only keys. - **Access:** any key. - **Cost:** Free; not rate limited. ### Arguments None. ### Result (`structuredContent`) | Field | Type | Required | Description | |---|---|---|---| | `object` | string | No | `list`. | | `data` | object[] | No | The same entries as `GET /models`. | ### Example ```bash curl https://ostra.testrelic.ai/mcp \ -H "Authorization: Bearer $OSTRA_API_KEY" \ -H 'Content-Type: application/json' \ -H 'Accept: application/json, text/event-stream' \ -H 'MCP-Protocol-Version: 2025-06-18' \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "ostra_list_models", "arguments": {} } }' ``` ```json { "content": [ { "type": "text", "text": "{\"object\":\"list\",\"data\":[{\"id\":\"ostra-lite\",\"name\":\"Ostra Decision\",\"status\":\"beta\",\"release\":\"v.beta.5\",\"tasks\":[{\"task\":\"triage\",\"name\":\"Failure triage\",\"description\":\"Classifies a failing test as flaky, product bug, test bug, environment or selector drift.\",\"mcpTool\":\"ostra_triage_failure\",\"rateLabel\":\"$0.50 / 1M failures\",\"outputRateLabel\":\"$0.00 / 1M decisions\",\"version\":\"1.4.0\",\"contentHash\":\"sha256:3f9a1c07d2e4b8a6\",\"gatePassed\":true,\"classes\":[\"flaky\",\"product_bug\",\"test_bug\",\"environment\",\"selector_drift\"],\"deferThreshold\":0.6},{\"task\":\"test_impact\",\"name\":\"Test impact\",\"description\":\"Ranks candidate tests by the probability that a diff breaks them.\",\"mcpTool\":\"ostra_select_impacted_tests\",\"rateLabel\":\"$2.00 / 1M diffs\",\"outputRateLabel\":\"$0.00 / 1M decisions\",\"version\":\"1.1.2\",\"contentHash\":\"sha256:a07be3315c9d4f12\",\"gatePassed\":true,\"classes\":[\"not_impacted\",\"impacted\"],\"deferThreshold\":0.55},{\"task\":\"selector_survival\",\"name\":\"Selector survival\",\"description\":\"Predicts whether UI locators survive the next release.\",\"mcpTool\":\"ostra_check_selector\",\"rateLabel\":\"$0.25 / 1M selectors\",\"outputRateLabel\":\"$0.00 / 1M decisions\",\"version\":\"1.2.0\",\"contentHash\":\"sha256:5d21f0e9c4a7b3e8\",\"gatePassed\":true,\"classes\":[\"survives\",\"breaks\"],\"deferThreshold\":0.6}],\"pricing\":{\"input\":\"$0.25–$2.00 / 1M inputs\",\"output\":\"$0.00 / 1M decisions\"},\"latencyP50Ms\":41,\"requests7d\":1284},{\"id\":\"ostra-pro\",\"object\":\"model\",\"created\":1794873600,\"owned_by\":\"testrelic\",\"name\":\"Ostra Pro\",\"version\":\"1.0\",\"base_url\":\"https://ostra.testrelic.ai/api/v1\",\"context_window\":262144,\"max_output_tokens\":32768,\"pricing\":{\"input\":\"$0.64 / 1M input tokens\",\"cached_input\":\"$0.06 / 1M cached input tokens\",\"output\":\"$4.47 / 1M output tokens\"},\"mcp_tool\":\"ostra_pro_chat\"}]}" } ], "structuredContent": { "object": "list", "data": [ { "id": "ostra-lite", "name": "Ostra Decision", "status": "beta", "release": "v.beta.5", "tasks": [ { "task": "triage", "name": "Failure triage", "description": "Classifies a failing test as flaky, product bug, test bug, environment or selector drift.", "mcpTool": "ostra_triage_failure", "rateLabel": "$0.50 / 1M failures", "outputRateLabel": "$0.00 / 1M decisions", "version": "1.4.0", "contentHash": "sha256:3f9a1c07d2e4b8a6", "gatePassed": true, "classes": [ "flaky", "product_bug", "test_bug", "environment", "selector_drift" ], "deferThreshold": 0.6 }, { "task": "test_impact", "name": "Test impact", "description": "Ranks candidate tests by the probability that a diff breaks them.", "mcpTool": "ostra_select_impacted_tests", "rateLabel": "$2.00 / 1M diffs", "outputRateLabel": "$0.00 / 1M decisions", "version": "1.1.2", "contentHash": "sha256:a07be3315c9d4f12", "gatePassed": true, "classes": [ "not_impacted", "impacted" ], "deferThreshold": 0.55 }, { "task": "selector_survival", "name": "Selector survival", "description": "Predicts whether UI locators survive the next release.", "mcpTool": "ostra_check_selector", "rateLabel": "$0.25 / 1M selectors", "outputRateLabel": "$0.00 / 1M decisions", "version": "1.2.0", "contentHash": "sha256:5d21f0e9c4a7b3e8", "gatePassed": true, "classes": [ "survives", "breaks" ], "deferThreshold": 0.6 } ], "pricing": { "input": "$0.25–$2.00 / 1M inputs", "output": "$0.00 / 1M decisions" }, "latencyP50Ms": 41, "requests7d": 1284 }, { "id": "ostra-pro", "object": "model", "created": 1794873600, "owned_by": "testrelic", "name": "Ostra Pro", "version": "1.0", "base_url": "https://ostra.testrelic.ai/api/v1", "context_window": 262144, "max_output_tokens": 32768, "pricing": { "input": "$0.64 / 1M input tokens", "cached_input": "$0.06 / 1M cached input tokens", "output": "$4.47 / 1M output tokens" }, "mcp_tool": "ostra_pro_chat" } ] }, "isError": false } ``` Reference page: https://docs.testrelic.ai/lores/api/mcp/ostra-list-models # Webhook events | Event | When | |---|---| | `billing.topup` | A top-up payment succeeded. | | `billing.auto_topup` | An auto top-up charged the saved card. | | `billing.auto_topup_failed` | An auto top-up charge failed. Three failures in a row turn auto top-up off. | | `billing.coupon_redeemed` | A coupon code added credit. | | `billing.spend_alert` | This month’s spend crossed the alert percentage of the monthly spend limit (once a month). | | `api_key.created` | A key was created. | | `api_key.revoked` | A key was revoked. | | `member.joined` | Someone joined the workspace. | | `member.removed` | A member was removed. | | `member.role_changed` | A member’s role changed. | | `mcp.tool_enabled` | An MCP tool was turned on. | | `mcp.tool_disabled` | An MCP tool was turned off. | | `webhook.test` | You clicked Send test event. | Each delivery is a JSON POST with headers `Ostra-Event`, `Ostra-Delivery` and `Ostra-Signature: t=,v1=.")>`. Details: https://docs.testrelic.ai/lores/api/webhooks