TestRelicGet Started with DeepEval
Install testrelic-deepeval, log in, run a DeepEval suite, and view the eval run in TestRelic.
Getting started takes three steps: install the package, log in, and run your DeepEval suite. The pytest plugin auto-registers, so there is no code to add — your existing DeepEval tests upload automatically.
Paste this prompt into your coding agent:
Add TestRelic reporting to this DeepEval project.
1. Install the package: pip install testrelic-deepeval
2. Authenticate by running `testrelic login` and entering my TestRelic API key,
or set the environment variable TESTRELIC_API_KEY to my key. Do not hardcode
it into source — use the shell environment or the project's secrets mechanism.
3. Do NOT add any plugin wiring or imports — the pytest plugin auto-registers
via its entry point.
4. Run `deepeval test run tests/` and confirm the eval run uploads to TestRelic.
5. Open the run with `testrelic view`.Full quickstart
pip install testrelic-deepeval
testrelic login
deepeval test run tests/There is nothing to import and no plugin wiring to add. As long as testrelic-deepeval is installed in the same environment as DeepEval, the pytest plugin captures the run automatically. With no API key present it simply no-ops — it never fails your tests.
Need DeepEval too? Install it with the optional extra: pip install "testrelic-deepeval[deepeval]".
The SDK talks to the TestRelic production endpoint by default and authenticates with your tr_* key — no endpoint configuration is needed. To target a different endpoint, set TESTRELIC_BASE_URL or TESTRELIC_CLOUD_ENDPOINT, or pass base_url= to the wrapper. See Configuration.
Requirements
- Python >=3.10 (tested 3.9–3.12)
- DeepEval
>=2.0,<4.0(installed via the[deepeval]extra, or separately)
Next steps
Package Overview
How testrelic-deepeval captures DeepEval runs — the pytest plugin, the evaluate() wrapper, the offline queue, and the native evals upload path.
Cloud Quickstart
Authenticate, run a DeepEval evaluation, and see it land in the TestRelic eval workspace, Test Runs feed, and repo Evaluations tab.