Playground

Try Ostra Pro and Ostra Decision in the TestRelic Console — chat, presets, thinking, tools, attachments, Compare, history, model status, and copying the request as code.

The Playground (console.testrelic.ai/playground) runs the Ostra models from your browser. It's the fastest way to see what a model does with your own failures, code and screenshots, and to get the exact request as code.

Playground runs are billed

Playground requests cost the same as API calls and count toward your rate limit. They show in Usage and Logs under the Console channel. Owners, admins and developers can run the Playground; other roles see "Your role can't change this. Ask an admin."

Pick a model

The model picker at the top switches between:

ModelShows
Ostra Pro 1.0Chat. "$0.64 in / $4.47 out per 1M tokens", with a live status: Ready, Starting or Asleep.
Ostra Decision BetaTasks. "from $0.25 in / $0.00 out per 1M", Always on.
Ostra CodeComing soon (not selectable).

The Playground opens on Ostra Pro. Links from Ostra Decision pages and from Logs → Open in playground open it on Ostra Decision.

Ostra Pro chat

Start a conversation

An empty chat asks What are you testing today? and offers:

  • Run a sample: eight ready-made problems with files to work on, such as "A test that fails only in some orders", "The first real failure in a 2,000-line CI log" or "Pick the tests a change can break".
  • Or start from your own: Write a Playwright test, Explain a failed CI run (opens Insert a failed run), Find why a test is flaky, Review a test diff, Turn a bug report into test cases, Check a screenshot for UI regressions.

Settings

The settings rail controls the request:

SettingValues
PresetFailure analyst (default), Test writer, Code reviewer, Custom. Each sets a system prompt.
System promptFree text. Editing it switches to Custom.
ThinkingOn or off. With it on, Reasoning effort is Low, Medium or Max.
Max output tokens1–32,768 (default 8,192), reasoning included. The rail shows "Up to $X per reply, reasoning included."
ToolsAdd tool, up to 16. Each has a Name (1–64 letters, digits, _ or -), a Description (up to 1,000 characters) and Parameters (JSON Schema).
More settingsTemperature (0–2) and Top-p (0–1).

When Ostra Pro calls one of your tools, a card shows the call. Paste the result and click Send result, or Skip this call. The sample problems come with their own tools that can Run in your browser (a sandbox with no network); turn on Run tools automatically to let them run on their own, up to 12 rounds per message ("Paused after 12 tool rounds", then Keep going). Sample files live in the browser tab only; Reset files restores them and Show the answer reveals the intended fix.

Send

  • Attach images (PNG, JPEG, WebP or GIF, up to 8 per conversation) and one video (MP4 or WebM), or drop or paste them. Attachments together can be about 5.2 MB.
  • Insert a failed run picks one of your Ostra Decision triage requests from the last 30 days and adds its details.
  • Send with Ctrl+Enter (⌘+Enter on Mac); Stop ends the reply.
  • Before you send, the composer shows the most the reply can cost and your balance: "Up to $X · balance $Y".

While Ostra Pro answers you see its reasoning ("Thinking…", collapsible) and a live cost ("about N tokens · $X so far"). Under a finished reply: token counts, cost and speed, plus Copy and Regenerate. A reply you stop says "Stopped after about N tokens. You're billed only for what it produced." with Continue.

Inspector

The inspector panel has three tabs:

  • Session: this conversation's cost; tokens (Input, Cached, Output) and what cached input saved; speed (First token, Output speed); model status; spend this month; Open this conversation in Logs.
  • Code: the current request as curl, Python or TypeScript (OpenAI SDK, with an x-ostra-session header so follow-up turns go to the same replica when possible and more often hit the prompt cache). Copy it into your project and set OSTRA_API_KEY.
  • JSON: the last request and response.

Compare

Compare sends the same prompt with two settings side by side: "Same prompt, two settings. Both replies are billed." Side B starts as a copy of A with Thinking flipped; Edit settings changes either side. Each side shows its settings and Cost, Output, First token and Total time, with badges such as Cheapest, Fastest and Reasons first, and a summary like "B costs 6.6× more · A answers 6.5× faster". The composer button becomes Send to both. Keep A's settings or Keep B's settings ends the comparison with the winner; Exit compare ends it unchanged.

History

History lists your conversations in this workspace (the latest 50), saved after each reply. Rename a conversation by clicking its title (up to 120 characters), or delete it ("Delete '…'? This can't be undone. Billing and Logs keep their records."). New chat starts over.

  • Conversations are kept for the workspace's log retention. With retention set to Zero retention, history is off ("History is off for this workspace").
  • A conversation can hold up to 400 messages and about 300 KB. Past that: "This conversation is too long to save to history. Start a new chat to keep saving."

Model status and warm-up

Ostra Pro scales to zero after 5 quiet minutes. The status pill shows Ready, Starting or Asleep. When it's asleep, the next request starts it, which usually takes under a minute and up to about 5 minutes the first time after a model update; the Playground shows "Starting Ostra Pro mm:ss". Warm up now (owners, admins, developers) starts it before you send. See Cold starts.

Ostra Decision tasks

Pick a task card: Failure triage, Test impact or Selector survival, each with its price.

  1. Load one of the Examples ("Timeout in CI", "Locator removed in diff", "Assertion after deploy", "Network 503"; "Checkout change", "Selector file change"; "Three locators", "Single test id"), or Insert a failed run (Load) from your recent triage requests.
  2. Edit the input. Failure triage has a Form (error class or "Derive from the message", framework, error message, test, runs and failures in the last 50) and a JSON editor; the other tasks use JSON. The input shapes are in Run a task.
  3. Click Run (Ctrl+Enter). The cost line reads, for example, "Costs $0.0000005 per failure · Same failure within 24 h: served from cache, free".

The result has three views: Decision (the verdict, confidence and next step), JSON (the raw response) and Code (the request as cURL, Python, TypeScript or MCP). Under it: time, cost (or "cached, free"), request ID and Open in Logs. A triage result also offers Ask Ostra Pro why, which opens Ostra Pro with the Failure analyst preset. Recent runs keeps your last five.

If a run fails, an error card explains why; a 402 adds Go to billing. See Errors.

Notices you may see

NoticeMeaningWhat to do
"This reply could cost up to $X"Your balance doesn't cover the reply's maximum cost.Buy credits, or Lower to N tokens to reduce Max output tokens.
"Too many requests right now… Sending again in N s"You hit the workspace's rate limit.Wait; it retries by itself.
"The answer stopped before it finished"The reply was cut off.Try again. You're billed only for what was produced.
"Starting Ostra Pro mm:ss"Ostra Pro is starting after a quiet period.Wait, or use Warm up now next time.
"Request failed (status)"Any other error.Check the status in Errors.
Was this page helpful?

On this page