By Vivek Trivedy, Product Manager @ LangChain
Evals are training data for Agents
In classical machine learning, training data guides the model’s learning process. Each training example contributes a gradient that updates the model’s weights toward “correctness.” We have a similar learning loop for agents.
Evals encode the behavior we want our agent to exhibit in production. They’re the "training d...
The article presents a compelling case for using evaluations ("evals") as a form of training data to refine AI agent behavior, drawing a clear parallel to classical machine learning. The strongest version of this narrative is that evals provide a structured way to encode desired behaviors, enabling iterative improvement through a feedback loop that includes human oversight and holdout sets to prevent overfitting. This approach is grounded in practical engineering principles, such as data quality...