Find the term.
See how it fits.
Explore the concepts behind AI agents, one useful connection at a time.
Concept directory
Agent Consistency Gap
The agent consistency gap is the difference between average success across repeated runs and the share of tasks that succeed on every run. It reveals instability that a single score can hide.
Benchmark
A benchmark is a defined set of tasks and scoring conventions used to compare performance. An AI benchmark measures selected capabilities under its particular test conditions.
Evaluation
Evaluation is checking an AI system against defined tasks and success criteria. An evaluation may score an answer, inspect actions, or test whether the resulting environment matches the intended outcome.
Evaluation Harness
An evaluation harness is software that runs evaluation tasks against models or systems and collects results. It connects task definitions, model execution, and scoring into a repeatable testing process.
LLM-as-a-Judge
LLM-as-a-judge uses a language model to assess outputs against instructions or a rubric. It can produce scores, comparisons, or written judgments to support evaluation.
Trace
A trace records the operations involved in an execution so their sequence and relationships can be inspected. In distributed tracing, individual operations are represented as spans with timing and other recorded attributes.