Changelog

Uploaded visual comparisons had nowhere to be reviewed. Now they have two.

• A Visual tab in the session workspace of every Playwright session: every comparison the run made, most changed first, with baseline and this run side by side, a wipe, an onion skin and Playwright's pixel diff. The Elements panel names the element that caused the change and draws its box over the render; the ten things that only shifted below it are filed as knock-on movement, not ten findings.
• A standalone Visual Regression workspace reached the way Web and Mobile are reached, for triaging a whole session's comparisons at once.
• Accept and reject, recorded. A decision is stored against the session with who made it and why; tr visual accept from the CLI lands in the same ledger, so every surface shows the same answer. A pass, a baseline that was only written, and a size mismatch cannot be decided — the workspace says so instead of offering a button that would be refused.
• One row per comparison. Retries and repeat uploads no longer add rows of their own. Each comparison is one row, and the last attempt is the verdict. A first run shows six snapshots as six rows, each marked baseline written, not twelve with three of them green.

The reporter learned to say which element caused a visual failure two days ago. This release closes the loop: the CLI and Studio show the same attribution, and a reviewer can accept or reject the new baseline without leaving the terminal — with a record of who decided what, and why.

• tr visual accept / tr visual reject. Accept copies this run's render over the committed baseline file the report names — the one Playwright reads next time — and nothing else; reject changes no files. Both refuse what cannot be decided (a pass, a baseline that was only written, a size mismatch), both record the decision as a line in <report>/visual-decisions.jsonl before anything touches the network, and both sync it to TestRelic Cloud when the run was uploaded there — otherwise one line says why it stayed local. --note travels with the decision. See Visual regression.
• tr visual lists a retried test once, with the final attempt's verdict, and names a comparison by snapshot, by <spec>::<snapshot>, or by test id — an ambiguous name is refused with the candidates listed, never guessed.
• Studio has a Visual tab in the cloud test stack, beside Steps, Console and Network: baseline against this run, side by side or as Playwright's pixel diff, the element list with the changed element's box drawn over the render, and Accept / Reject wired to the platform. The same tab appears for a report opened from disk, without the decision buttons — nothing leaves the machine there.
• The step cursor that scopes the panels is set from the Steps tab in the cloud test stack.

Released as tr 0.103.0 and the matching Studio build. Accept needs a report from @testrelic/playwright-analytics 2.16.2 or later, which records where each committed baseline lives.

A first run of a visual suite records nothing it can compare — Playwright writes the baselines and reports each test as failed. 2.16.0 listed every one of those assertions twice, one of them green. This release makes the verdicts true and gives a reviewer the path to act on them.

• One assertion, one record, the right word. toMatchVisualBaseline reads the committed file to decide what happened: a first write is new, a rewrite under --update-snapshots is updated (and the rewritten baseline is attached, so the report has a picture of it), and a match is passed. Two assertions resolving at once can no longer report each other's verdict. (Under updateSnapshots: 'none' a missing baseline fails the test and Playwright attaches nothing for it, so the report does not list that comparison — unchanged here.)
• A native toHaveScreenshot failure stays visible even when it shares a name with a toMatchVisualBaseline pass in the same test.
• Each comparison records where its committed baseline lives — baselinePath, relative to your Playwright config's directory — so tr visual accept promotes a render into the file Playwright actually reads, not the per-run copy in the report. (The platform records the decision; only the CLI, which has your checkout, writes the file.)
• Merged shard reports keep their comparisons and their images. A merged report used to claim comparisons nothing could show.
• The report uses the platform's words — Wipe, Pixel diff, This run — so a comparison reads the same in the HTML report, in Studio and in the cloud.

Released as @testrelic/playwright-analytics 2.16.2 and @testrelic/core 2.9.1. Report schema moves to 1.7.0; the addition is optional.

Four things were captured on every run and never reached the report. They are in it now.

• The device. The report showed you two facts about the device — android · native. There is now a device strip: OS and version, device name, driver, app under test, native or webview mode, UDID. Rows and the detail header wear a device badge. The platform has a field of its own now, so it is no longer appended to your own tag list.
• Step categories. A swipe, an app launch, a keypress and a screenshot all rendered as the same badge. Steps are badged by kind now, and the run is broken down by it. The cloud timeline and Ask AI filters are unchanged.
• Device-log provenance. A crash-log line used to be indistinguishable from ordinary logcat chatter. Each line now keeps its tag, source, pid and stack trace, and shows them.
• Every screenshot, not just the first. One screenshot per failure has always been captured, but only one reached the report. All of them are kept in capture order and rendered as a strip.

There is also a Replay view — the screen recording with a scrubber marked once per step, where clicking a step seeks the video — and the shared run-level layout: a verdict band, tiles for what this run actually captured, and a needs-attention block that leads with the failing step and the device-log line recorded beside it.

Two attribution fixes: screenshots are scoped per test, and AI defects belong to the flow they were found in rather than to the run. Network entries stop claiming every request is xhr and derive a resource type from the response MIME type.

Released as @testrelic/appium-analytics 1.2.0. The report schema moves to 1.1.0 — every addition is optional, so anything reading 1.0.0 keeps working.

Four things every Maestro run already captured never appeared in the report. They do now.

• Replay. The screen recording with a scrubber marked once per step; clicking a step seeks the video.
• Driver log. maestro.log was never rendered in the report at all. It has a Driver log tab, with each line's source, tag, pid and stack trace.
• The device. The device a run ran on never reached the report. It now opens with a device strip, and flow rows carry a device badge. The platform is a field of its own, so it is no longer appended to your tag list.
• Step kinds. Ten kinds of Maestro step all rendered as one of four badges, so AI and DEV steps never showed a badge of their own. Steps are badged by kind now.

The run overview also gains the shared layout: a verdict band saying what broke, tiles for what the run actually captured, and a needs-attention block that leads with the failing step and the driver-log line recorded beside it.

Screenshots and AI defects are now scoped to the flow they belong to; every flow used to show the whole run's screenshots.

Released as @testrelic/maestro-analytics 1.4.0. The report schema moves to 1.1.0 — every addition is optional.

A pixel diff can tell you 80,858 pixels moved. It cannot tell you which card gained a margin. This release captures a DOM snapshot alongside every screenshot and diffs it against a committed baseline, so the report names the elements responsible.

• Visual comparisons appear in the report. If your suite already uses toHaveScreenshot() or toMatchSnapshot(), upgrade the reporter and they show up — baseline, actual and diff, viewable side by side, through a wipe slider, as an onion-skin blend, or as Playwright's own diff image, with the differing-pixel count and the exact ratio. No import to add, no assertion to rewrite. TestRelic does not compare images itself; Playwright already ships the comparator, so this adds nothing to your dependency tree.
• toMatchVisualBaseline records comparisons that passed. Native toHaveScreenshot attaches nothing when images match, which makes a green visual check invisible to any reporter. The new matcher runs the same comparator with the same options and additionally records the baseline it asserted against.
• DOM comparison names what changed. Element geometry, a curated set of computed styles, visible text and identity attributes are captured, diffed, and reported as seven kinds of change — added, removed, moved, resized, restyled, text, and attrs. Hovering a row in the new Elements tab draws that element's box over the screenshot, so the semantic change and the pixels it caused are connected. Needs Playwright 1.51 or later; everything else in the reporter still runs on 1.35.
• Knock-on movement is separated from the actual edit. One card gaining a 24px margin shifts everything below it. A flat list would report eleven equal findings for one change, so elements that only moved or resized are grouped beneath the element that did something.
• DOM baselines are text, and belong in git. A <name>.dom.json sits beside each image baseline, so a restyle arrives in a pull request as margin-top: 24px in a readable diff rather than an opaque binary change. --update-snapshots rewrites both files together, and the mask option you already pass excludes regions from the DOM comparison as well as the pixel one.
• The run overview leads with what broke. A verdict band opens the page, and a Needs attention block promotes failures to the top with their cause, their evidence and the pixel diff itself. Test rows carry evidence chips saying what was captured and a duration bar relative to the slowest test on screen. The test detail is now a page with its own URL and history entry, so a link to a failing test opens that test.
• API assertions are captured again. The documented setup — importing test from /fixture and expect from /api-fixture — passed every assertion through uncaptured. Our own example suite reported 0 assertions; it now reports 126. There is a new Assertions section on the detail page, failures first, each with its expression, expected and received values, source line, and the call it was made against.
• Media that was already on disk is reachable. Screenshots and video in a report opened from disk now load and play. Traces now have a tab of their own, with the npx playwright show-trace invocation.

Released as @testrelic/playwright-analytics 2.16.0 and @testrelic/core 2.9.0. Report schema moves to 1.6.0 — every addition is optional.

See Visual Testing for the full guide, including how to keep a visual suite from going flaky.

When something goes wrong, Studio now tells you what happened and how to fix it.

• Every error comes with steps. A short explanation, then what to do, in order. The technical detail goes to a log file instead of filling the dialog.
• Every error has a page. Click More help in the message to jump straight to the matching section of the new Troubleshooting Studio page.
• Two common failures are named properly now. Antivirus software that inspects HTTPS traffic can break Studio's connection — Studio recognises that and retries on its own. And a mistyped server address used to be blamed on antivirus, which no antivirus setting could fix.

• A chat that reached its session token cap used to stop there permanently. It now compacts and carries on — the cap triggers compaction rather than ending the conversation.
• Token spend that was quietly under-reported is now counted in full, so usage reflects what was actually consumed.
• An organization can be given its own Ask AI session token cap and spend ceiling instead of sharing one platform-wide default. The two move as a pair, so a ceiling raised alone can't silently change which limit you hit.