Skip to main content
Ask AI

Configuration

testrelic-deepeval is configured through environment variables, the credentials file, or explicit arguments. You rarely need more than testrelic login to get started — this page covers the full set of options.

Environment variables​

VariablePurpose
TESTRELIC_API_KEYYour TestRelic API key
TESTRELIC_BASE_URLOverride the cloud base URL
TESTRELIC_PROJECTProject identity to attach the run to
TESTRELIC_REPO_GIT_IDRepository git identity for the run

Create an API key from your org — see API keys.

Authentication precedence​

When the SDK resolves a setting, it uses the first source that provides a value:

  1. Explicit arguments (for example, CLI flags or evaluate() keyword arguments)
  2. Environment variables
  3. Credentials file (~/.testrelic/credentials.toml)
  4. Defaults

Credentials file​

testrelic login writes credentials to ~/.testrelic/credentials.toml (mode 0600 on POSIX). Remove them with testrelic logout.

Base URL override​

The cloud endpoint defaults to https://platform.testrelic.ai/api/v1/evals in the final release. (Pre-release builds default to stage.) Override it with TESTRELIC_BASE_URL or at login:

Terminal
testrelic login --api-key <key> --base-url <url>

Project and repo identity​

Use TESTRELIC_PROJECT and TESTRELIC_REPO_GIT_ID to control which project and repository a run is attached to. The SDK also auto-detects CI context — branch, commit, and CI URL — for GitHub Actions, GitLab, Jenkins, CircleCI, and Buildkite, so these are usually inferred automatically in CI.

Offline queue location​

Undeliverable uploads are written to ~/.testrelic/queue/ and replayed with testrelic drain.

evaluate() wrapper options​

The programmatic wrapper is a drop-in for deepeval.evaluate():

from testrelic import evaluate

results = evaluate(test_cases, metrics, **kwargs)
  • test_cases — your DeepEval test cases.
  • metrics — the DeepEval metrics to evaluate against.
  • **kwargs — forwarded to deepeval.evaluate().

Authentication and identity follow the same precedence rules above. Explicit keyword arguments take priority over environment variables and the credentials file. The return value is whatever deepeval.evaluate() returns; the upload happens automatically.

Next steps​