Use the full-page Ask AI chat to get answers about your test suite, generate reports, plan test coverage, and create structured artifacts using natural language.

Ask AI is the primary conversational AI interface in TestRelic. It provides a full-page chat experience where you can ask questions about your test suite, request analysis, and generate structured artifacts — all using natural language.

Your prompt

Ask AI chat

Org test context

Answers and artifacts

Growth plan required

Ask AI requires the Growth plan. See the Cloud Platform section for plan details.

Accessing Ask AI

Click Ask AI in the left sidebar navigation.
The URL format is /ai for a new conversation or /ai/:conversationId to resume an existing conversation.

Starting a conversation

Type your question or request in the input box and press Enter. The AI responds with a streaming reply — text appears token-by-token as it is generated.

Example prompts

Analyze recent failures

What are the most common failure patterns in the last 7 days
across the @staging repository?

Generate a test plan

Create a test plan for the checkout flow covering happy path,
payment failure, and out-of-stock scenarios.

Ask about a specific run

Summarize the failures in @run-20240315 and suggest which ones
are likely infrastructure issues vs real bugs.

Attaching context with @ mentions

The @ context picker lets you attach specific entities to your message so the AI can answer with precision. Supported context types:

Context type	What it is
`@repo`	A specific repository
`@test_run`	A specific test run by ID
`@test_case`	A specific test case
`@branch`	A Git branch
`@suite`	A test suite grouping
`@tag`	A test tag label
`@environment`	A deployment environment
`@integration`	A connected integration

Type @ in the input box to open the context picker and search for the entity you want to attach.

TestRelic Cloud/ai · Ask AI

Checkout flake triage

Weekly quality report — payments

Coverage gaps in signup flow

What are the most common failure patterns in the last 7 days across @web-app?

Called get_failure_patterns · web-app, 7d

Three patterns dominate the last 7 days: OAuth redirect timeouts (14 sessions, all before the payment iframe mounts), toast-timing assertions failing only in CI, and two selectors broken by the v3 cart migration.

web-app

Ask about your tests — type @ to attach a repo, run, branch, or app…

Brain = the Memory toggle (on by default) · click it in the live composer to try

Conversation history

All conversations are saved automatically. Each one is given an auto-generated topic title based on its content, so the sidebar stays scannable without any manual naming. The Chat Sidebar on the left of the Ask AI page lists all your previous conversations. From there you can:

Resume any past conversation.
Rename a conversation for easier reference.
Delete conversations you no longer need.

Conversations are private to your user account within the organization. Remote sessions are the exception: their owner can share them with the workspace (see Share a session).

Remote sessions

Ask AI conversations running in Studio or the CLI, and sessions running on your own runners, appear in Ask AI on the web as Remote sessions. From the web you can follow them live, reply, answer approvals, start new ones and share them with your workspace. The device or runner still runs every turn itself, with its own tools and permission rules.

The Remote tab

A Chats / Remote switcher heads the Ask AI sidebar. Chats holds your web conversations. Remote lists sessions under Live now, Shared with me and Archived, which holds the ones that have ended. Search works inside whichever list you're on.

There's nothing to set up: conversations from Studio and the CLI appear on their own.

A session reads like Studio's own chat:

The header says where the agent works, for example "runs and edits on device in repo".
Answers are headed TestRelic, with how long the agent worked. Consecutive tool calls fold into one "N actions · click to expand" line.
Only what was typed. A message sent from Studio or the CLI shows the person's own words. The instructions those apps add to it, such as the reply language and the project rules from TESTRELIC.md, are hidden. Attached files and recorded steps show as chips under the message.

Reply, approve and stop

Reply or queue a message. The composer reads "Reply to device…". While a turn is running, a message you send waits in a queue: Send now sends a held message, and Remove drops it.
Answer approvals. When the session is waiting on an approval, answer it with Yes, run it, No, skip this or No — tell it what to do instead. Once the card has focus, keys 1–3, y and n, the arrow keys, Enter and Esc work too. The approval is also open on the device, and whichever answer comes first is used. "Don't ask again" and Autopilot can only be turned on there.
Review a save before you approve it. When the device asks before saving a file, Review changes on the approval card opens the change in the Artifact pane. You can answer Yes, run it or No, skip this there; to tell it what to do instead, answer on the card.
Stop a turn. Stop this turn interrupts the device's current turn, and the session stays open. A runner session can also be ended with End session: the runner stops and deletes its checkout of your repository, and the conversation stays in Ask AI, where you can continue it as a web chat.

Replying from the web needs Studio or the CLI 1.2.1 or later. Sessions from older versions can still be watched, and the session asks you to update.

Composer options

Permission. The permission pill switches between the device's own approval setting ("Studio’s setting", for example) and ⏸ Plan first: click it, press Shift+Tab or type /permission. Under Plan first, that turn gets only the device's read-only tools (reading files and folders, the docs, and git status, diff, log and branches), so it plans and changes nothing. Write tests freely and Full autopilot are chosen on the device.
The + menu has Attach a file, for text files under 2 MB (a file longer than 12,000 characters is cut, and the agent is told so), and Reference a test or run, which works like typing @.
/ commands: /permission, /model, /effort and /memory, plus prompts for reports, dashboards, plans, decks, comparisons and alerts.
Chat settings choose the model and reasoning effort for the messages you send, and Memory marks one message to be remembered. Remote sessions always answer with TestRelic Cloud.

Code and changes

Long code folds into a card. A code block of more than three lines becomes a one-line card with its language or file name and line count, and Open shows it in the Artifact pane. Shorter blocks stay in the chat with line numbers, a wrap toggle and Copy. On a test file, Save & run writes that request into the composer.
A card for every file the agent touched. Each file it saved, deleted or diffed on the device gets a card with its path, the lines added and removed, and a create, update or delete badge.
Code and Changes. The pane shows the code with syntax highlighting, and what changed as a diff with old and new line numbers. A save is compared with the last copy of that file the chat saw. When there isn't one, for example because the chat never read the file or the read stopped at Studio's 64 KB limit, the pane opens on the code and says why.
Step through the chat's code with the previous and next arrows, and copy or download it. Ask about this code puts a question about it in the composer.

Start a session from the web

New Session on the Remote tab opens New remote session:

Your devices lists the Studio apps you're signed in to. Pick a recent folder and say what the agent should do; the chat opens and runs in Studio on that computer.
A runner from Infrastructure opens Run on infrastructure, which starts the session on a runner you've set up in CI or on a cloud machine. See Infrastructure for setting up a runner.

Share from the session header. The session's owner gets Share, next to the people who have access and who's watching now.
Two levels of access. Add workspace members by name or email and give each Can view or Can collaborate. Or set general access to Everyone in your workspace, who can then view it (the default is Only people added). Copy link gives a link that opens only for members with access.
Viewers see the live session, its transcript and its artifacts. They can Request to collaborate, and the owner can Allow or Dismiss the request in the Share dialog.
Collaborators can send messages. A collaborator's message runs on the owner's device with the owner's tools and permissions, and the transcript shows who sent it ("Sent by … from the web").
What stays with the owner: approvals, stopping a turn, ending the session, and changes to the conversation itself, such as renaming or deleting it.
Finding what's shared with you. When someone adds you to a session, a notification appears, and sessions shared with you are listed under Shared with me. Remove from my list hides one without changing anyone's access. If you haven't seen a share in the app within 5 minutes, you get one email about it.

Only members of the same workspace can be given access, and someone who leaves the workspace loses it.

Control from Studio and the CLI

Whoever runs the session decides whether the web can take part:

In Studio, each chat's header has a Web control on / Web control off switch, and messages sent from the web carry a From the web label. Under Settings → Chat, the TestRelic web app panel has Web control in new chats, which sets the default, and Start chats from the web, which decides whether the web may open chats on this computer. Both are on by default.
In the CLI, turn web control off with --no-remote-control, the remoteControl config key or the TESTRELIC_REMOTE_CONTROL environment variable, or mid-session with /remote off (and back on with /remote on).

With web control off, the web can still watch the session.

Message feedback

You can rate any AI response using the thumbs up / thumbs down feedback controls on each message. This feedback helps improve the AI's responses over time.

AI artifacts

Some requests cause the AI to generate a structured artifact in addition to the text response. Artifacts appear in a dedicated panel to the right of the chat. The supported artifact types are:

Artifact type	Description
`dashboard`	A rendered analytics dashboard with charts and metrics
`report`	A formatted test quality or coverage report
`test_plan`	A structured test plan with scenarios and steps
`presentation`	A slide-style summary for sharing with stakeholders
`code`	A code snippet (e.g. a new test, a reporter config)
`data_table`	A tabular breakdown of test data
`chart`	An inline data visualization
`navigation_paths`	A test navigation path map

See AI Insights & Artifacts for more detail on individual artifact types.

Ask AI's default artifact template is designed for every width an artifact is shown at, from the full page down to the side panel and a phone. Its stat tiles go from four a row to two and then one, and its wide tables scroll inside their own box. Screenshots embedded in an artifact show one at a time, fitted whole, with previous and next arrows and a counter.

Any artifact can also be shared via a public or private link (Google-Drive-style): public links render a read-only view of the artifact — with its filters — to anyone who has the link, while private links require org sign-in. You can also grant specific people access to an individual artifact, the same way you'd share a file with named collaborators in Google Drive. Shared artifacts are also collected in an artifact library. See sharing artifacts via links.

Streaming and tool use

Behind the scenes, the AI uses tool calling to fetch data from your repositories, runs, and test cases. While a tool call is running, the UI shows an indicator. The result of the tool call is incorporated into the final AI response — you do not need to take any action.

The streaming connection uses Server-Sent Events (SSE). If the connection is interrupted, the platform automatically retries and resumes streaming.

For multi-step requests, Ask AI shows a live plan as it works: a step-by-step task list that updates in place as each step starts and completes, so you can follow the agent's progress in real time rather than waiting for one large answer. The plan is especially visible in the CLI, where the running task list renders inline in the terminal.

Answering from the TestRelic docs

Ask AI can read the TestRelic documentation to answer product and onboarding questions — for example "how do I configure the reporter?" or "what does the coverage score mean?" — alongside questions about your own test data. This lets a single conversation move between "what does my suite look like?" and "how do I set this up?" without leaving the chat.

Memory

Ask AI can remember context from past sessions so that follow-up conversations feel continuous. Memory is controlled by a toggle in the composer bar — it is on by default.

Not the same as repo Memory

This section covers conversation memory — per-user recall of your recent chats and preferences. The team-shared, per-repository knowledge base of test decisions and constraints is a separate feature: see Memory.

How memory works

Memory operates in three layers. The layers are independent and stack on top of each other:

Layer	What it contains	Always active?
Current thread	The last 20 messages of the open conversation, loaded from the database	Yes — unaffected by the toggle
Recent conversation history	Short excerpts pulled from your other recent conversations across all threads	Only when Memory is on
Long-term memory	Durable facts about your preferences, recurring topics, and working patterns, extracted by the AI and stored across sessions	Only when Memory is on and enabled on your instance

When Memory is on, all three layers are combined into the system context sent to the AI before each reply. When you turn Memory off, only the current thread history is used.

Enabling and disabling memory

The Memory toggle appears:

In the Ask AI composer bar at the bottom of the full-page chat.
In the AI Assistant panel (the floating assistant available on every page).

Toggle it off for any conversation where you want the AI to respond without prior-session context — for example, when troubleshooting an isolated issue or starting fresh on a new project.

Memory is scoped to your user account. Colleagues in the same organization cannot see or access your memory context.

Ask AI Apps

Ask AI Apps are third-party service integrations — such as Slack, Gmail, GitHub, and Jira — that you connect to Ask AI. Once connected, the AI can call actions in those services directly from the chat: look up Slack threads, draft emails, open Jira issues, and more.

Connecting an app

Go to Settings → Integrations → Ask AI Apps.
Browse the available app catalog and click the app you want to connect.
Complete the OAuth authorization flow for that service.
The app appears as Connected in the list and is now available in Ask AI.

You can manage connected apps (view status, disconnect) from the same page at any time.

Using an app in a conversation

Once an app is connected, you can bring it into any Ask AI conversation in two ways:

@ context picker — Type @ in the composer, search for the app by name, and select it from the list. The app is attached to your message as a context item.
Apps control in the composer bar — Click the apps icon in the composer bar to open the apps picker, then select the connected app.

After attaching an app, the AI can call its tools when it determines an action is relevant to your request. You can attach multiple apps to a single message.

How app tool calls work

When an app is attached, the AI is given access to that app's tools. The call is transparent: the UI shows a tool-call indicator while the action runs, and the result is incorporated into the AI's reply.

Connecting a Jira app alongside your test context lets Ask AI open tickets, query existing issues, and link failures directly — without leaving the chat.

Workspace vs personal connections

Some apps support a workspace scope (shared across your organization's Ask AI sessions) and a personal scope (visible only to you). The scope is shown when you initiate the connection and can be reviewed on the Ask AI Apps settings page.

TestRelic Slack App

The TestRelic Slack App is the inverse of Ask AI Apps: instead of the AI calling third-party services from the web chat, the Slack App lets your team trigger Ask AI from inside Slack. Both can be enabled at the same time — they're complementary, not exclusive.

Feature	Ask AI Apps	TestRelic Slack App
Where you use it	Web app composer	Any Slack channel or DM
Direction	AI → third-party tool	Slack user → Ask AI
Typical use	"Open a Jira ticket for this failure"	"/testrelic which tests broke after deploy?"
Memory continuity	Per Ask AI conversation	Per TestRelic user (Slack ↔ web is unified)

See the Integrations section for the full Slack reference. The summary below covers what's specific to Ask AI behavior.

Slash commands

Type /testrelic <prompt> in any channel the app is a member of. The prompt runs through the same Ask AI runtime as the web composer — same tool calls, same context, same artifact types — and the response threads under your message:

Channel: #qa-leads

/testrelic which tests broke after yesterday's deploy?

Channel: #checkout-team

/testrelic generate a sprint review deck for @repo:web-app covering the last 14 days

Threaded follow-ups continue the same Ask AI conversation, so context like @repo or @test_run set on the first prompt carries through.

Scheduled QA digests

Any Ask AI prompt can be scheduled to post recurring summaries to a Slack channel — weekly flaky digests, Monday sprint reviews, nightly regression reports. Each digest is a Slack card with metric tiles, top offenders, and a deep link back to the live artifact in the web app.

Create a digest by running the prompt once in the web composer and clicking Share → Schedule to Slack, or from Slack with:

/testrelic schedule a weekly flaky digest to this channel every Monday 9am

Manage all scheduled digests at Settings → Integrations → Slack → Schedules.

Every artifact rendered in the Ask AI panel has a Share → Slack action. Pick a channel, add an optional message, and the artifact posts as a rich card with metric snapshots and a deep link. Presentations and reports upload as Slack files so they're searchable later.

Connecting Slack

Install from Settings → Integrations → Slack in the web app, or from the Slack App Directory. Full install steps, OAuth scopes, and troubleshooting live in the Integrations section.

Pairing with Jira

Combine the Slack App with the Jira integration and an Ask AI Apps Jira connection to triage and file tickets entirely from Slack: /testrelic was this a flake or regression? → /testrelic open a Jira ticket for this failure with the failure context attached.

TestRelic Microsoft Teams App

The TestRelic Microsoft Teams App brings Ask AI into Teams with full parity to the Slack App: run a prompt in a channel, schedule recurring QA digests, share artifacts back from the web composer, get quality-gate failure alerts, and continue the same conversation through threaded follow-ups. The same Ask AI runtime, artifacts, memory, and privacy posture apply. Teams-originated conversations appear in the Chat Sidebar with a Teams badge and are unified with your web and Slack conversations.

Installation is a two-step flow, and this is where Teams differs from Slack:

Connect (OAuth) at Settings → Integrations → Microsoft Teams → Connect. This uses Microsoft Entra ID (Azure AD) and binds your Teams tenant to your TestRelic org.
Add the Teams app package — sideload it yourself or have a Teams admin upload it to your org's app catalog (and enable the Azure Bot Teams channel).

Connect alone does not add the bot

Unlike Slack, OAuth Connect only links the tenant. The TestRelic bot appears in Teams only after the app package is added in step 2. If the bot is missing after Connect, the package has not been installed yet.

See the Integrations section for the full Microsoft Teams install steps, command usage, digests, failure alerts, and troubleshooting.

Ask AI