跳转至

apogee-ai-eval

Offline + online evaluation harness with G-Eval, faithfulness, bias and judge-LLM scorers.

This page is being expanded

The tables below are generated from the package source, so they are accurate. Narrative guides and worked examples are being written module by module.

Public API

Everything below is exported from the package root, so from apogee_ai_eval import X works for each name.

Layer Symbols
Application · DTOs CompareRunsDTO, GateDTO, GateResultDTO, RunSuiteDTO, SetBaselineDTO
Application · Use cases CompareRunsUseCase, GetRunUseCase, ListRunsUseCase, RegressionGateUseCase, RunSuiteUseCase, SetBaselineUseCase
Domain Baseline, CaseComparison, CaseInvocation, CaseResult, Comparison, EvalCase, EvalRun, EvalSuite, JudgeVerdict, MetricComparison, MetricResult, MetricSpec, RegressionGate, RegressionStatus, Score, Severity, Verdict
Domain · Enums MetricKind
Domain · Exceptions BaselineNotFoundException, CaseNotFoundException, DatasetParseException, EvalError, InvokerException, JudgeException, MetricException, RegressionGateException, RunNotFoundException, SuiteNotFoundException
Domain · Protocols (ports) IAgentInvoker, IBaselineRepository, IDatasetRepository, IJudge, IMetric, IReporter, IRunRepository, ISuiteRepository
Infrastructure AnthropicJudge, ContainsMetric, CostMetric, DeepEvalAdapter, EchoAgentInvoker, EqualsMetric, FunctionAgentInvoker, InMemoryBaselineRepository, InMemoryDatasetRepository, InMemoryRunRepository, InMemorySuiteRepository, JsonBaselineRepository, JsonMatchMetric, JsonReporter, JsonRunRepository, JsonlDatasetRepository, JudgeCustomMetric, JudgeFaithfulnessMetric, JudgeRelevanceMetric, JudgeToxicityMetric, JunitReporter, LatencyMetric, LengthMetric, MarkdownReporter, OpenAIJudge, RagasAdapter, RegexMetric, RuleBasedJudge, TrajectoryMetric, TruLensAdapter, YamlSuiteRepository, default_metrics

Install

Bash
pip install apogee-ai-eval

The base install has no required third-party dependency. Each extra unlocks one adapter.

Extra Pulls in
anthropic anthropic>=0.25
openai openai>=1.30
ragas ragas>=0.2, datasets>=2.20
trulens trulens-eval>=1.0
deepeval deepeval>=2.0
all anthropic>=0.25, openai>=1.30, ragas>=0.2, datasets>=2.20, trulens-eval>=1.0, deepeval>=2.0

CLI

Installed as a console script, and registered as an apogee plugin so the same subcommands work under the global binary.

Bash
apogee-eval --help