aws-ai-ml

An agent skill by aws, from aws/agent-toolkit-for-aws. Tags: analysis, cloud, deployment, ml, training.

What it does

Selects, deploys, and customizes AI models on Amazon SageMaker. Fine-tuning (SFT, DPO, RLVR, RLAIF), model selection, dataset preparation, evaluation, deployment to SageMaker endpoints or Bedrock, inference optimization and endpoint diagnostics. Covers the full lifecycle from planning through production. Use when fine-tuning models on SageMaker, choosing/selecting which base model to customize or fine-tune from SageMaker Hub, finding a model to deploy without fine-tuning, transforming datasets for training, checking data readiness, evaluating model quality, deploying to endpoints, benchmarking or optimizing inference, setting up IAM roles and S3 buckets for training jobs, or managing a SageMaker Managed MLflow app. Also use to check endpoint health, diagnose failures, debug latency or errors, or view container logs and CloudWatch metrics. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3. NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure.

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add aws/agent-toolkit-for-aws --skill aws-ai-ml

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/aws/agent-toolkit-for-aws
cp -r agent-toolkit-for-aws/plugins/aws-core/skills/aws-ai-ml ~/.claude/skills/aws-ai-ml

Safety box score

Grade D, elevated risk: 58 of 100 risk points, rated 2026-10-14. The grade rates what the skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. How the score works.

This skill covers end-to-end SageMaker AI/ML workflows: model selection, fine-tuning, evaluation, deployment to SageMaker or Bedrock, endpoint diagnostics, processing jobs, and SageMaker Managed MLflow app management. The main risks are that it runs code and AWS CLI/SDK actions, can create/update/delete cloud resources, and includes a HuggingFace download path that handles access tokens and remote network transfers.

Category levels, from 0 (none) to 3 (high)
Category	Level	Why
Code & shell execution	3 (high)	The skill ships runnable Python scripts and notebook templates that execute SDK and CLI operations, including `import boto3`, `sm_client.describe_hub_content(...)`, and multiple `%pip install ...` cells.
Network access	3 (high)	It makes outbound AWS API calls throughout the workflows and can also reach HuggingFace directly, e.g. `download_huggingface_model(...)` is explicitly described as a non-AWS network call to `huggingface.co`.
Filesystem reach	1 (low)	Most writes are scoped to the project directory (for example `Path("[PROJECT_DIR]") / "manifests"`) or container-local processing paths like `/opt/ml/processing/...`; there is no evidence of broad host-file access.
Secrets & credentials	2 (moderate)	The HuggingFace optimization path accepts `HF_HUB_TOKEN` and warns to prefer a Secrets Manager ARN, and the SDK setup resolves the caller's execution role from the authenticated environment.
Privilege & persistence	1 (low)	The skill does not require sudo or service registration, but it does direct the user to install packages (`pip install`, `%pip install`) and to grant/manage IAM roles and policies for SageMaker/Bedrock workflows.
Instruction hijack surface	0 (none)	I did not find prompt-injection style instructions to ignore safeguards, hide actions, or follow untrusted remote text; the explicit-confirmation language is safety-oriented.
Irreversible actions	3 (high)	It can create, update, and delete AWS resources on the user's behalf, including MLflow apps (`create-mlflow-app`, `update-mlflow-app`, `delete-mlflow-app`) and deployed models/endpoints via `model_builder.deploy(...)`.
Transparency	2 (moderate)	A large set of core files could not be read due size/budget, including `references/model-selection/overview.md`, `references/model-deployment/overview.md`, `references/finetuning/overview.md`, `references/model-evaluation/overview.md`, and several code templates/scripts.

Before installing

Read `references/manage-mlflow/references/app-deletion-workflow.md` and `app-update-workflow.md` if you do not want an agent to change or delete MLflow apps.
Review the deployment templates in `references/model-deployment/code_templates/` for `pip install`, AWS SDK calls, and endpoint creation behavior.
If you will use the HuggingFace flow, plan to supply `HF_HUB_TOKEN` from a secret store, not as a plain text variable.
Test in a scratch AWS account or region first, because this skill can create training jobs, endpoints, and MLflow apps.

Rated 2026-10-14 by an automated review of 69 files from the skill's repository.

Source

Repository: aws/agent-toolkit-for-aws (all skills from this repository)
Path: plugins/aws-core/skills/aws-ai-ml/SKILL.md
Branch: main
Collection: aws-core
Updated: 2026-09-19

amazon-bedrock — Builds generative AI applications on Amazon Bedrock.
aws-database — Routes any task involving AWS databases — choosing, comparing, recommending, getting started with.
aws-deployment — Configures CI/CD pipelines using AWS CodePipeline, CodeBuild, CodeDeploy, CodeConnections, and CodeArtifact.
agents-deploy — Use when deploying your agent to AWS, or when a deploy has failed.
chatting-with-aws-devops-agent — Have a fast, conversational analysis with the AWS DevOps Agent.
coordinating-multi-space-devops-agent — Coordinate the AWS DevOps Agent across multiple AgentSpaces from one Claude Code session — route questions to the right space (prod vs staging vs knowledge).