Curated resources for validating RAG systems, retrieval quality, grounding, faithfulness, hallucinations and data reliability.
Validation
community
AI & ML interests
AI validation for models, agents, data, tools and autonomous systems.
Recent Activity
View all activity
Curated resources for validating agent behavior, tool use, trajectories, recovery and autonomous AI systems.
-
Agent Validation
🧭Validate agent behavior, tool use, and recovery.
-
MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use
Paper • 2512.24565 • Published -
Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench
Paper • 2604.16706 • Published • 1 -
An Empirical Study of Automating Agent Evaluation
Paper • 2605.11378 • Published • 2
Curated tools and research on AI validation, model evaluation, agent reliability, testing and reproducible AI systems.
Curated resources for AI validation methods, benchmark design, testing protocols and reliable evaluation workflows.
-
Validation Readiness
✅Assess AI validation readiness across key system layers.
-
CheckEval: Robust Evaluation Framework using Large Language Model via Checklist
Paper • 2403.18771 • Published -
A Unified Framework for the Evaluation of LLM Agentic Capabilities
Paper • 2605.27898 • Published -
Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI
Paper • 2607.22368 • Published • 1
Curated resources for validating model quality, robustness, reliability, calibration and reproducible AI evaluation.
-
Model Validation
🧪Validate model quality, robustness, and reliability.
-
Brittlebench: Quantifying LLM robustness via prompt sensitivity
Paper • 2603.13285 • Published -
Lessons from the Trenches on Reproducible Evaluation of Language Models
Paper • 2405.14782 • Published • 1 -
Benchmark Agreement Testing Done Right: A Guide for LLM Benchmark Evaluation
Paper • 2407.13696 • Published • 5
Curated resources for validating RAG systems, retrieval quality, grounding, faithfulness, hallucinations and data reliability.
Curated resources for AI validation methods, benchmark design, testing protocols and reliable evaluation workflows.
-
Validation Readiness
✅Assess AI validation readiness across key system layers.
-
CheckEval: Robust Evaluation Framework using Large Language Model via Checklist
Paper • 2403.18771 • Published -
A Unified Framework for the Evaluation of LLM Agentic Capabilities
Paper • 2605.27898 • Published -
Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI
Paper • 2607.22368 • Published • 1
Curated resources for validating agent behavior, tool use, trajectories, recovery and autonomous AI systems.
-
Agent Validation
🧭Validate agent behavior, tool use, and recovery.
-
MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use
Paper • 2512.24565 • Published -
Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench
Paper • 2604.16706 • Published • 1 -
An Empirical Study of Automating Agent Evaluation
Paper • 2605.11378 • Published • 2
Curated resources for validating model quality, robustness, reliability, calibration and reproducible AI evaluation.
-
Model Validation
🧪Validate model quality, robustness, and reliability.
-
Brittlebench: Quantifying LLM robustness via prompt sensitivity
Paper • 2603.13285 • Published -
Lessons from the Trenches on Reproducible Evaluation of Language Models
Paper • 2405.14782 • Published • 1 -
Benchmark Agreement Testing Done Right: A Guide for LLM Benchmark Evaluation
Paper • 2407.13696 • Published • 5
Curated tools and research on AI validation, model evaluation, agent reliability, testing and reproducible AI systems.