Triage a failure

The ostra_triage_failure tool on the Ostra MCP server — arguments, result, errors and examples.

MCPhttps://ostra.testrelic.ai/mcp · ostra_triage_failure

Classifies a failing test as flaky, product bug, test bug, environment or selector drift, with a confidence.

Title
Ostra Decision: Failure triage
Access
Full-access key. Read-only keys can list the tool but not call it.
Cost
$0.50 per 1M failures, same as REST. Cached answers are free.
Hints
readOnlyHint: true · destructiveHint: false · idempotentHint: true · openWorldHint: false

Once the Ostra MCP server is connected, your agent calls the tool with these arguments.

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "ostra_triage_failure",
    "arguments": {
      "test_id": "checkout.spec.ts > applies coupon",
      "run_id": "gha-18241",
      "context": {
        "framework": "playwright",
        "capture_level": "full",
        "error_message": "TimeoutError: locator.click: Timeout 30000ms exceeded.\n  waiting for locator('#apply-coupon')",
        "action": "click",
        "locator": {
          "strategy": "css"
        },
        "diff": {
          "files_changed": 3,
          "lines_added": 41,
          "lines_removed": 12,
          "touches_test_file": false,
          "touches_selector_file": false,
          "locator_values_removed": false
        },
        "dom": {
          "locator_found": true
        },
        "memory": {
          "runs_50": 40,
          "flake_rate_50": 0.15,
          "failed_count_50": 6,
          "signature_repeat_count": 5
        }
      }
    }
  }
}
Headers
mcp-protocol-version
2025-06-18
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "{\"id\":\"req_01JDXK4M7Q2N8R5T6V9W3Y1Z0A\",\"model\":\"ostra-lite@v.beta.5\",\"task\":\"triage\",\"cached\":false,\"billed\":{\"units\":1,\"costNano\":500,\"costUsd\":\"0.0000005\"},\"output\":{\"decision_id\":\"dec_5b0c9e2a7f3d4e18a6b2c9d0e1f4a7b3\",\"task\":\"triage\",\"decision\":\"flaky\",\"value\":\"flaky\",\"p\":0.83,\"probabilities\":{\"flaky\":0.83,\"product_bug\":0.06,\"test_bug\":0.05,\"environment\":0.04,\"selector_drift\":0.02},\"threshold\":0.6,\"defer_reason\":null,\"decided_by\":\"model\",\"rule\":null,\"model\":{\"release\":\"v.beta.5\",\"version\":\"1.4.0\",\"content_hash\":\"sha256:3f9a1c07d2e4b8a6\",\"gate_passed\":true},\"test_id\":\"checkout.spec.ts > applies coupon\",\"run_id\":\"gha-18241\",\"error_class\":\"timeout\",\"error_class_source\":\"error_message\"}}"
      }
    ],
    "structuredContent": {
      "id": "req_01JDXK4M7Q2N8R5T6V9W3Y1Z0A",
      "model": "ostra-lite@v.beta.5",
      "task": "triage",
      "cached": false,
      "billed": {
        "units": 1,
        "costNano": 500,
        "costUsd": "0.0000005"
      },
      "output": {
        "decision_id": "dec_5b0c9e2a7f3d4e18a6b2c9d0e1f4a7b3",
        "task": "triage",
        "decision": "flaky",
        "value": "flaky",
        "p": 0.83,
        "probabilities": {
          "flaky": 0.83,
          "product_bug": 0.06,
          "test_bug": 0.05,
          "environment": 0.04,
          "selector_drift": 0.02
        },
        "threshold": 0.6,
        "defer_reason": null,
        "decided_by": "model",
        "rule": null,
        "model": {
          "release": "v.beta.5",
          "version": "1.4.0",
          "content_hash": "sha256:3f9a1c07d2e4b8a6",
          "gate_passed": true
        },
        "test_id": "checkout.spec.ts > applies coupon",
        "run_id": "gha-18241",
        "error_class": "timeout",
        "error_class_source": "error_message"
      }
    },
    "isError": false
  }
}

Arguments

  • test_idstring

    Your identifier for the failing test, echoed back in the answer. Use a stable name, e.g. checkout.spec.ts > applies coupon.

    1–512 chars
  • run_idstring

    Your identifier for the CI run, echoed back.

    1–512 chars
  • contextobjectRequired

    Evidence about the failure. Every field is optional except that error_class or error_message must be present. Missing evidence is treated as missing, never as zero, so send what you have.

    Show 10 child fields
    • idstring

      Your own id for this failure record.

      1–512 chars
    • frameworkstring

      Test framework that produced the failure.

      One ofplaywrightcypressjestselenium
    • capture_levelstring

      How much of the failure was captured (trace, DOM, network): full, partial or none.

      One offullpartialnone
    • error_classstring

      Normalized class of the error. Required unless you send error_message.

      One oftimeoutlocator_not_foundstale_elementstrict_mode_violationassertion_mismatchnetwork_errorscript_errorunmapped
    • error_messagestring

      The raw error text. Used only to derive error_class when you don’t send one; the answer reports which you used in error_class_source.

      0–8,000 chars
    • actionstring

      The step that failed.

      One ofclickfillnavigateassertwait_forapi_callscreenshot
    • locatorobject

      The locator the failing step used.

      Show 1 child field
      • strategystring

        How the element was located.

        One ofcssxpathtest_idtextrolenone
    • diffobject

      The code change the run tested.

      Show 6 child fields
      • files_changedinteger

        Files changed in the commit or pull request under test.

        0 to 1,000,000
      • lines_addedinteger

        Lines added.

        0 to 1,000,000
      • lines_removedinteger

        Lines removed.

        0 to 1,000,000
      • touches_test_fileboolean

        The change edits a test file.

      • touches_selector_fileboolean

        The change edits a page object, locator map or other file that defines selectors.

      • locator_values_removedboolean

        The change removes or renames a locator value (an id, test id, class or text) that the test uses.

    • domobject

      What the page looked like at failure time.

      Show 1 child field
      • locator_foundboolean

        The locator still matched an element in the captured DOM.

    • memoryobject

      This test’s recent history, over its last 50 runs at most.

      Show 6 child fields
      • runs_50integer

        Runs in the window.

        0 to 50
      • failed_count_50integer

        Failed runs in the window.

        0 to 50
      • passed_on_retry_50integer

        Runs that failed and then passed on retry.

        0 to 50
      • consecutive_failures_50integer

        Failures in a row up to this one.

        0 to 50
      • flake_rate_50number

        Share of runs in the window that were flaky, 0–1.

        0 to 1
      • signature_repeat_countinteger

        How many times this exact error signature has been seen before.

        0 to 1,000,000

Result

content[0].text holds the answer as text; structuredContent holds it as JSON:

  • idstring

    Request id.

  • modelstring

    Model and release, e.g. ostra-lite@v.beta.5.

  • taskstring

    The task.

  • cachedboolean

    Answered from the 24-hour cache.

  • billedobject

    What this request cost. Cached answers cost 0.

    Show 3 child fields
    • unitsinteger

      Billable units: 1 per failure (triage), 1 per request (test impact), 1 per locator (selector survival). 0 for a cached answer.

    • costNanointeger

      Cost in nano-dollars (1 USD = 1,000,000,000).

    • costUsdstring

      Cost in USD as an exact decimal string, e.g. "0.0000005".

  • outputobject

    The task’s answer, the same shape as /predict.

Errors

Billing, validation, rate-limit and permission problems come back as a normal result with isError: true and structuredContent.error (type, message), the same types as the REST errors. Authentication failures are HTTP 401/403 before any JSON-RPC.

Was this page helpful?