benchmark-models

An agent skill by garrytan, from garrytan/gstack. Tags: evaluation, ml, multi-agent.

What it does

Cross-model benchmark for gstack skills. (gstack)

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add garrytan/gstack --skill benchmark-models

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/garrytan/gstack
cp -r gstack/benchmark-models ~/.claude/skills/benchmark-models

Safety box score

Grade D, elevated risk: 56 of 100 risk points, rated 2026-10-16. The grade rates what the skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. How the score works.

This skill runs a cross-model benchmark by sending the same prompt to Claude, GPT/Codex, and Gemini, then optionally a judge model. The main risks are network exposure of prompt content, touching local auth material in home-dotfiles, and delegating key behavior to opaque helper scripts.

Category levels, from 0 (none) to 3 (high)
Category	Level	Why
Code & shell execution	2 (moderate)	It shells out to `gstack-skill-start`, `gstack-model-benchmark`, `gstack-skill-end`, `gstack-learnings-log`, plus `mkdir`, `chmod`, and `mktemp`; that is a real command-running workflow, though not arbitrary code execution.
Network access	2 (moderate)	It explicitly runs the same prompt through Claude, GPT/Codex, and Gemini and logs telemetry, so prompt content and results leave the machine to remote services.
Filesystem reach	2 (moderate)	It writes benchmark artifacts under `~/.gstack/tmp/` and `~/.gstack/benchmarks/`, updates git exclude, and reads from `~/.claude/.credentials.json`, so it has home-directory side effects.
Secrets & credentials	2 (moderate)	It checks `ANTHROPIC_API_KEY` and `~/.claude/.credentials.json` and tells the user to log in to provider accounts, so it touches authentication material.
Privilege & persistence	0 (none)	No sudo, service registration, shell startup edits, or global installs are shown; `chmod 700` is only applied to a temp directory.
Instruction hijack surface	2 (moderate)	It tells the agent to treat the skill file as executable instructions and to follow `GSTACK_INSTRUCTION` blocks from tool output, which trusts fetched runtime text.
Irreversible actions	2 (moderate)	It automatically dispatches benchmark runs and can enable a paid judge pass, but only after explicit user choices and without destructive actions.
Transparency	2 (moderate)	Core behavior is delegated to helper programs at `~/.claude/skills/gstack/bin/gstack-skill-start`, `gstack-model-benchmark`, and `gstack-skill-end`, whose implementations are not included here.

Before installing

Inspect `SKILL.md` for every path under `~/.claude/` and `~/.gstack/` before running it on private prompts.
Open `~/.claude/skills/gstack/bin/gstack-model-benchmark` and the start/end scripts to verify what they send to providers and telemetry endpoints.
Test once with a non-sensitive prompt and confirm which providers are authed before using real project data.
If you do not want home-dotfile access, do not install this skill in the same account that holds your provider credentials.

Rated 2026-10-16 by an automated review of 1 file from the skill's repository.

Source

Repository: garrytan/gstack (all skills from this repository)
Path: benchmark-models/SKILL.md
Branch: main
Updated: 2026-10-05

arena — Spawn N parallel candidates at the same task, pick a base, graft the strongest parts of the losers into it.
marketing-council — When the user wants multiple expert perspectives on a marketing question — a simulated board of advisors staffed by legendary marketers (Seth Godin.
fine-tuning-expert — Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks.
llm-eval-harness — Test/evaluate any LLM behind an OpenAI- or Anthropic-compatible endpoint: availability (max_tokens-aware).
ml-pipeline — Designs and implements production-grade ML pipeline infrastructure: configures experiment tracking with MLflow or Weights & Biases.
rag-architect — Designs and implements production-grade RAG systems by chunking documents, generating embeddings, configuring vector stores, building hybrid search pipelines.