evaluation

An agent skill by muratcankoylan, from muratcankoylan/Agent-Skills-for-Context-Engineering. Tags: analytics, automation, developer-tools, evaluation, performance.

What it does

This skill should be used when building agent evaluation systems: deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, baseline comparison, and outcome measurement for agent pipelines.

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add muratcankoylan/Agent-Skills-for-Context-Engineering --skill evaluation

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/muratcankoylan/Agent-Skills-for-Context-Engineering
cp -r Agent-Skills-for-Context-Engineering/skills/evaluation ~/.claude/skills/evaluation

Safety box score

Grade A, minimal risk: 4 of 100 risk points, rated 2026-09-22. The grade rates what the skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. How the score works.

This skill is a mostly self-contained evaluation framework: documentation plus a Python demo script that scores agent outputs, manages test sets, and tracks production-style monitoring in memory. It has very low machine risk because it does not read or write user files, touch secrets, call networks, or require elevated privileges; the main effect is local computation and printing, with any gating behavior only arising when a user integrates it into their own pipeline.

Category levels, from 0 (none) to 3 (high)
Category	Level	Why
Code & shell execution	1 (low)	`scripts/evaluator.py` is a runnable Python module with a `if __name__ == "__main__":` demo and several executable classes, but it only performs local in-process logic.
Network access	0 (none)	No code or instructions perform HTTP requests, downloads, uploads, or other remote communication; the script only uses standard-library modules like `time` and `random`.
Filesystem reach	0 (none)	The files do not read, modify, or delete project or home-directory files; the demo only prints results and stores samples in memory.
Secrets & credentials	0 (none)	There is no access to `.env` files, tokens, keychains, browser sessions, or other credential stores, and no prompts asking the user to paste secrets.
Privilege & persistence	0 (none)	Nothing requests `sudo`, global installs, services, shell startup edits, or other privilege changes.
Instruction hijack surface	0 (none)	The documentation contains normal usage guidance only; it does not instruct the agent to ignore higher-priority instructions, hide actions, or follow remote content.
Irreversible actions	0 (none)	The repo evaluates and reports but does not itself push, deploy, post, pay, book, or delete anything.
Transparency	0 (none)	All shipped content is readable source text; there are no binaries, base64 blobs, downloaded installers, or unpinned remote code paths.

Before installing

Read `scripts/evaluator.py` to confirm you want a heuristic, in-memory evaluator rather than a real agent runner.
If you plan to integrate it into CI, test the pass/fail thresholds on a scratch pipeline first so you understand the gating behavior.
Verify the default rubric and sample-set size fit your use case; the shipped demo uses only three tests.

Rated 2026-09-22 by an automated review of 3 files from the skill's repository.

Source

Repository: muratcankoylan/Agent-Skills-for-Context-Engineering (all skills from this repository)
Path: skills/evaluation/SKILL.md
Branch: main
Updated: 2026-09-19

agents-optimize — Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization.
advanced-evaluation — This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration.
context-compression — This skill should be used when long-running agent sessions need context compression, structured summarization, compaction, token-per-task optimization.
context-degradation — This skill should be used for diagnosing and mitigating context degradation: lost-in-middle failures, context poisoning, context clash, context confusion.
context-optimization — This skill should be used for improving context efficiency: context budgeting, observation masking, prefix or KV-cache strategy, partitioning.
latent-briefing — This skill should be used when the user asks to "share memory between agents", "KV cache compaction for multi-agent", "orchestrator worker context".