Slow test suites, slow feedback: making CI test runs worth waiting for

Every test suite starts fast. Then it grows a test at a time — each one individually reasonable — until one day the E2E job takes 40 minutes, developers merge without waiting for it, and failures get discovered after the decisions they were supposed to inform. A slow suite doesn't just cost compute; it changes team behavior. Once "kick it off and switch tasks" becomes the norm, you've lost the thing CI feedback was for.

The usual response — throw more parallel workers at it — helps until it doesn't. Here's the more durable sequence: measure what's slow, fix the real sinks, parallelize what's safe, and finally stop running everything on every commit.

Step 0: know where the time goes

Teams optimizing a slow suite usually guess, and usually guess wrong. The wall-clock of a CI job hides the distribution: three 4-minute tests, one serial project bottlenecking twelve idle workers, a beforeEach that logs in through the UI two hundred times.

You need per-test duration over time, not one run's numbers. A single run tells you what was slow today; a trend tells you what's getting slower — the test that crept from 20s to 90s over a quarter is invisible in any snapshot. The TestRelic reporter records per-test and per-navigation timing with every run, and history in the dashboard turns that into exactly this view. Ask AI makes triage a sentence: "which ten tests contribute the most total runtime, and which regressed the most this month?"

The navigation-level timing matters more than it sounds: a slow test is usually not slow everywhere. The per-navigation network stats show whether the time went into one endless third-party script, one slow API call, or forty small waits — three very different fixes.

The classic sinks (fix these before buying runners)

• UI login in every test. Authenticate once, save storage state, reuse it. This single change is routinely worth minutes.
• Fixed sleeps. Every waitForTimeout(5000) is five seconds of pure waste times every test times every run. Replace with event- or state-based waits (and they were causing flakiness anyway — see the flaky-tests guide).
• UI setup for non-UI state. Creating a product by clicking through the admin panel takes 30 seconds; creating it via API takes 300ms. Test one flow through the UI; set up all other state through the API.
• Serial bottlenecks. One test.describe.serial block or one shared account forcing workers: 1 on a project can idle the rest of the fleet. (Shared accounts are a data isolation problem — fixing that unlocks parallelism as a side effect.)

Parallelism and infrastructure, second

Once tests are independent, scale-out is nearly free: more workers, sharding across CI machines, and it works — because you did the isolation homework first. Parallelizing a suite with shared state just converts slowness into flakiness, which is a worse trade.

Where the runs happen matters too. Local machines and shared CI runners have wildly different performance envelopes, and duration comparisons across them are noise. TestRelic records CI metadata with every run, so you can compare like with like — and execution infrastructure gives suites a consistent, managed place to run so a "regression" isn't just a cold runner.

The endgame: run the right tests, not all of them

Past a certain suite size, no amount of optimization makes "everything on every commit" fast. The mature move is tiering:

• Smoke on every push — a small, fast, ruthlessly stable set guarding the critical paths.
• Regression on merge / schedule — the broad suite, where ten extra minutes costs little.
• Nightly for the expensive stuff — cross-browser sweeps, long scenarios, performance runs.

Two things make tiering work in practice rather than on a wiki page. First, visibility per tier: TestRelic's monitoring views treat smoke, regression, and nightly as separate lanes with separate baselines, so a smoke failure is a page-someone signal while a nightly failure is a triage-tomorrow signal. Second, choosing what belongs in the fast tier from evidence: run history tells you which tests actually catch regressions and which haven't failed meaningfully in six months — and the MCP server's impact capability lets an AI assistant reason over which tests matter for a given change, so the fast lane stays both fast and worth trusting.

A realistic sequence

1. Baseline total and per-test duration from history; find the top-ten time sinks.
2. Kill the classic sinks: cached auth, no fixed sleeps, API-based setup.
3. Fix data isolation, then raise parallelism until runners — not tests — are the limit.
4. Tier the suite; wire each tier to its own monitoring lane.
5. Alert on duration trends, not just failures — a suite that got 30% slower this quarter did so one innocent-looking test at a time.

The goal isn't an arbitrary number; it's behavioral. Feedback fast enough that developers wait for it gets acted on. Feedback slower than that gets discovered later, in production, at several times the price.