Evaluation & Testing

Mock everything your AI app talks to — LLM APIs, MCP, A2A, vector DBs, search. One package, one port, zero dependencies.

The self-improving Agentic QA harness with Memory. Write tests in natural language.
 Catch regressions before releases ship.

A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.

A test runner for agentskills.io-style AI agent skills

Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表，主要面向基础大模型评测，旨在探求生成式AI的技术边界.

Awesome papers involving LLMs in Social Science.

Deliver safe & effective language models

Deliver safe & effective language models

Free, open-source AWS emulator. LocalStack alternative: 26 services, 1,924 operations, 100% conformance. No account, no auth token, no paid tier.

Data-Driven Evaluation for LLM-Powered Applications

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root cause.

AI-powered E2E testing for 10 platforms. 253 MCP tools. Zero config. Works with Claude, Cursor, Windsurf, Copilot. Test Flutter, React Native, iOS, Android, Web, Electron, Tauri, KMP, .NET MAUI — all from natural language.

A comprehensive set of LLM benchmark scores and provider prices. (deprecated, read more in README)

Build, Improve Performance, and Productionize your LLM Application with an Integrated Framework

A list of LLMs Tools & Projects

Python SDK for running evaluations on LLM generated responses

Computer-Use SDK for E2E QA Testing

QA Skills Directory QA Skills is a curated directory of testing-specific skills for AI coding agents (Claude Code, Cursor, Copilot, etc.).

Awesome QA Skills — a bilingual (zh/en) AI testing Agent Skills library for Codex, Cursor, Claude Code, Kiro, OpenCode, and Trae. Ships 4 testing workflows and 25 testing-type skills (58 skill folders with language parity): independently installable, composable, and eval-ready with skill-up. Covers requirements, strategy, cases, API/performance/sec

AI接口自动化测试 Skill：面向 Python + pytest + requests，驱动 Codex / Claude Code 生成、维护和调试接口用例（AI API test automation skill for Python + pytest + requests）

Spec2Case 是生产级 AI 测试用例生成智能体，支持图片/文本需求理解、人工确认、LangGraph 流程编排和 Excel 用例导出。

50 QA and test-automation skills for Claude Code, Codex, Cursor, and any Agent Skills Standard runtime.

XMind Vault