fine-tuning-expert

An agent skill by Jeffallan, from Jeffallan/claude-skills. Tags: api, evaluation, generative, ml, mlops, training.

What it does

Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add Jeffallan/claude-skills --skill fine-tuning-expert

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/Jeffallan/claude-skills
cp -r claude-skills/skills/fine-tuning-expert ~/.claude/skills/fine-tuning-expert

Safety box score

Grade D, elevated risk: 46 of 100 risk points, rated 2026-10-20. The grade rates what the skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. How the score works.

This skill is a full fine-tuning and deployment playbook for LLMs: it prepares datasets, trains LoRA/QLoRA adapters, evaluates models, and exports/serves them. The main risks are executing opaque model/repo code during loading or export, sending evaluation data to external APIs, and publishing fine-tuned artifacts to remote services.

Category levels, from 0 (none) to 3 (high)
Category	Level	Why
Code & shell execution	3 (high)	The deployment examples load models with `trust_remote_code=True` and shell out with `subprocess.run([...])`, which can execute repository-controlled Python or native binaries.
Network access	2 (moderate)	The skill includes outbound service use such as `OpenAI()`/`client.chat.completions.create(...)`, `push_to_hub(...)`, and remote model downloads via `from_pretrained(...)`.
Filesystem reach	1 (low)	Its examples write checkpoints/adapters to local output directories and delete `model-f16.gguf`, but the filesystem impact is mostly confined to the working tree/output path.
Secrets & credentials	2 (moderate)	External API calls rely on ambient credentials (`OpenAI()`, Hugging Face hub pushes), and evaluation data can include sensitive prompts, outputs, or labels.
Privilege & persistence	0 (none)	No sudo, service installation, shell startup edits, or other host-privilege changes are shown in the shipped files.
Instruction hijack surface	0 (none)	There is no hidden text or instruction to ignore safeguards; the prompt text in the examples is ordinary model-evaluation content.
Irreversible actions	2 (moderate)	The skill can publish and serve models (`push_to_hub`, vLLM server, Kubernetes manifests), which is an inherent deployment capability but still an outward action.
Transparency	2 (moderate)	Behavior depends on opaque external code paths such as `trust_remote_code=True` and `LLAMA_CPP_PATH`, so a reviewer cannot fully inspect what will run.

Before installing

Inspect `references/deployment-optimization.md` for `trust_remote_code=True` and `subprocess.run(...)` before using third-party model repos.
Decide whether you want `OpenAI()`-based judging to send prompts/references off-device.
Check whether `push_to_hub(...)` or public serving (`--host 0.0.0.0`) matches your data-sharing policy.
Run the workflow in a scratch project with dummy data first, then review the generated training/eval artifacts before reuse.

Rated 2026-10-20 by an automated review of 6 files from the skill's repository.

Source

Repository: Jeffallan/claude-skills (all skills from this repository)
Path: skills/fine-tuning-expert/SKILL.md
Branch: main
Updated: 2026-09-19

amazon-bedrock — Builds generative AI applications on Amazon Bedrock.
ml-pipeline — Designs and implements production-grade ML pipeline infrastructure: configures experiment tracking with MLflow or Weights & Biases.
book-sft-pipeline — This skill should be used for book-to-SFT pipelines: ePub extraction, literary segmentation, author-voice dataset construction, style-transfer training.
llm-eval-harness — Test/evaluate any LLM behind an OpenAI- or Anthropic-compatible endpoint: availability (max_tokens-aware).
stepfun-tts — Generate Chinese / Japanese speech with StepFun's Contextual TTS — default `stepaudio-2.5-tts` (blind-judged better on neutral/emotive preset voices).
arena — Spawn N parallel candidates at the same task, pick a base, graft the strongest parts of the losers into it.