ab-testing

An agent skill by coreyhaines31, from coreyhaines31/marketingskills. Tags: ab-testing, analytics, evaluation, optimization, testing.

What it does

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics. For page-level conversion optimization, see cro.

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add coreyhaines31/marketingskills --skill ab-testing

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/coreyhaines31/marketingskills
cp -r marketingskills/skills/ab-testing ~/.claude/skills/ab-testing

Safety box score

Grade A, minimal risk: 4 of 100 risk points, rated 2026-09-23. The grade rates what the skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. How the score works.

This skill is markdown-only guidance for planning and analyzing A/B tests; it does not ship code, run commands, or modify systems. Its main reach is limited to reading optional local context files and pointing the agent to external sample-size calculators, so the blast radius is low.

Category levels, from 0 (none) to 3 (high)
Category	Level	Why
Code & shell execution	0 (none)	No shell commands, code evaluation, or native scripts are included; the repo is documentation only.
Network access	0 (none)	The skill only cites public calculator links as references and does not instruct automatic downloads or data exfiltration.
Filesystem reach	1 (low)	It explicitly tells the agent to read optional workspace files like `.agents/product-marketing.md`, `.claude/product-marketing.md`, or `product-marketing-context.md` before asking questions.
Secrets & credentials	0 (none)	It does not ask for tokens, passwords, `.env` files, browser sessions, or other credentials.
Privilege & persistence	0 (none)	There is no sudo use, global install, service registration, or OS/git setting change.
Instruction hijack surface	0 (none)	I found no instruction to ignore higher-priority rules, hide behavior, or follow remote instructions.
Irreversible actions	0 (none)	It advises on experimentation design and analysis only; it does not send messages, deploy, pay, book, or delete anything.
Transparency	0 (none)	All shipped files are plain markdown; there are no binaries, minified blobs, or runtime-decoded content.

Before installing

Review `SKILL.md` for the optional local-file lookup behavior if you want to avoid exposing adjacent workspace notes.
Skim `references/sample-size-guide.md` and `references/test-templates.md` so you know the built-in assumptions and templates it will use.
If your environment restricts outbound browsing, note the external calculator links in `references/sample-size-guide.md` before relying on them.

Rated 2026-09-23 by an automated review of 4 files from the skill's repository.

Source

Repository: coreyhaines31/marketingskills (all skills from this repository)
Path: skills/ab-testing/SKILL.md
Branch: main
Collection: marketingskills
Updated: 2026-09-21

cro — When the user wants to optimize, improve, or increase conversions on any marketing page or form — including homepage, landing pages, pricing pages.
ads — When the user wants help with paid advertising campaigns on Google Ads, Meta (Facebook/Instagram), LinkedIn, Twitter/X, or other ad platforms.
aso — When the user wants to audit or optimize an App Store or Google Play listing.
attribution — When the user wants to figure out which marketing actually drives conversions and revenue, choose or interpret an attribution model.
onboarding — When the user wants to optimize post-signup onboarding, user activation, first-run experience, or time-to-value.
paywalls — When the user wants to create or optimize in-app paywalls, upgrade screens, upsell modals, or feature gates.