Evals

Studio's workbench rail adds an Evals surface, next to Live Browser and Live Device, for repos that run evals through the DeepEval SDK. It reads results straight off disk, so it works before or without ever uploading a run to the cloud platform's LLM Evaluations view.

How it finds your runs

The pane does a bounded walk of the open repo for DeepEval test-run JSON: .deepeval/, deepeval-results/, a DEEPEVAL_RESULTS_FOLDER export, or any run JSON it recognizes by shape (testCases + metricsData, in either the JS SDK's camelCase or the Python SDK's snake_case). The scan prunes node_modules, .git, target, and venv, and caps depth, file size, and run count, so it stays fast even in a large monorepo. Runs are listed newest first.

If nothing is found, the pane shows an empty state that explains how to produce a run rather than looking broken.

What you see

• Hero band — a pass-rate donut gauge (green/amber/red by health), plus stat blocks for test count, metrics passed (with the run's average score), total LLM-as-judge cost, and evaluation time.
• Metric health by category — one threshold-ticked score bar per DeepEval taxonomy bucket (RAG, Agentic, Multi-Turn, Safety & Compliance, Multimodal, Custom), each with its pass count and average threshold. Empty categories are skipped. A G-Eval/DAG metric that shares a name with a built-in still counts as Custom, since the judge is your own rubric, not the built-in one.
• Pass rate across runs — a sparkline over the repo's run history, oldest to newest.

• Case detail — every case shows the metric that most explains its result (the failing metric with the worst score-vs-threshold margin, or the tightest-margin pass on an otherwise clean case), a per-metric verdict strip, and the full metric card with score, threshold, judge reason, evaluation model, and cost.
• Agent conversation and tool calls — for each case, the single-turn input/output or multi-turn transcript the metrics were scored against, plus the tool calls (with their input/output) the agent made along the way — so a failing Task Completion or Knowledge Retention score can be read against what the agent actually said and did.

{"order_id":"48213"}

{"status":"delivered","total":"$62.00","eligible_for_refund":true}

Next steps

→ DeepEval SDK overview · LLM Evaluations (cloud platform) · Workspace