Get Started with DeepEval

Getting started takes three steps: install the package, log in, and run your DeepEval suite. The pytest plugin auto-registers, so there is no code to add — your existing DeepEval tests upload automatically.

For AI agents

Paste this prompt into your coding agent:

Setup prompt

Add TestRelic reporting to this DeepEval project.
1. Install the package: pip install testrelic-deepeval
2. Authenticate by running `testrelic login` and entering my TestRelic API key,
   or set the environment variable TESTRELIC_API_KEY to my key. Do not hardcode
   it into source — use the shell environment or the project's secrets mechanism.
3. Do NOT add any plugin wiring or imports — the pytest plugin auto-registers
   via its entry point.
4. Run `deepeval test run tests/` and confirm the eval run uploads to TestRelic.
5. Open the run with `testrelic view`.

For Human

1. Install the package:

   Terminal

   pip install testrelic-deepeval

2. Log in (saves credentials to ~/.testrelic/credentials.toml):

   Terminal

   testrelic login

3. Run your DeepEval suite as you normally would:

   Terminal

   deepeval test run tests/

4. Open the run in your browser:

   Terminal

   testrelic view

Full quickstart

Terminal

pip install testrelic-deepeval
testrelic login
deepeval test run tests/

note

There is nothing to import and no plugin wiring to add. As long as testrelic-deepeval is installed in the same environment as DeepEval, the pytest plugin captures the run automatically. With no API key present it simply no-ops — it never fails your tests.

tip

Need DeepEval too? Install it with the optional extra: pip install "testrelic-deepeval[deepeval]".

Requirements

• Python >=3.10 (tested 3.9–3.12)
• DeepEval >=2.0,<4.0 (installed via the [deepeval] extra, or separately)

Next steps

• Cloud Quickstart
• CLI Reference
• Configuration
• Examples