Package Overview
testrelic-deepeval (pip name testrelic-deepeval, version 0.1.2) captures DeepEval evaluation results and uploads them to TestRelic's native /api/v1/evals/* API. It reads DeepEval's in-memory TestRun only — no monkey-patching and no HTTP redirect.
Install
pip install testrelic-deepeval
To install DeepEval alongside the SDK, use the optional extra (it pulls deepeval>=2.0,<4.0):
pip install "testrelic-deepeval[deepeval]"
| Detail | Value |
|---|---|
| pip name | testrelic-deepeval |
| Version | 0.1.2 |
| Python | >=3.10 (tested 3.9–3.12) |
| Core deps | httpx>=0.27, pydantic>=2.6, typer>=0.12, platformdirs>=4.0 |
| DeepEval extra | testrelic-deepeval[deepeval] → deepeval>=2.0,<4.0 |
The pytest plugin
The package ships a pytest plugin (entry point testrelic.deepeval.plugin) that auto-registers — there is nothing to import or configure.
- Auto-capture. At
pytest_sessionfinish, it captures DeepEval's globalTestRunand uploads it. - No-op without credentials. If no API key or credentials are present, it logs a message and does nothing.
- Never fails your tests. Upload exceptions are swallowed, so a reporting problem can never turn a green suite red.
The evaluate() wrapper
For programmatic evaluations outside pytest, import the wrapper:
from testrelic import evaluate
evaluate(test_cases, metrics, **kwargs) is a drop-in for deepeval.evaluate(): it runs the eval and then auto-uploads the result. The return value is whatever deepeval.evaluate() returns.
Upload path and offline queue
Results are uploaded to TestRelic's native /api/v1/evals/* API. The cloud endpoint defaults to https://platform.testrelic.ai/api/v1/evals in the final release. (Pre-release builds default to stage.)
If an upload cannot be delivered, it is written to a local offline queue at ~/.testrelic/queue/. Replay queued uploads later with testrelic drain.