spark-engineer

An agent skill by Jeffallan, from Jeffallan/claude-skills. Tags: analytics, cloud, data, mlops, performance.

What it does

Use when writing Spark jobs, debugging performance issues, or configuring cluster settings for Apache Spark applications, distributed data processing pipelines, or big data workloads. Invoke to write DataFrame transformations, optimize Spark SQL queries, implement RDD pipelines, tune shuffle operations, configure executor memory, process .parquet files, handle data partitioning, or build structured streaming analytics.

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add Jeffallan/claude-skills --skill spark-engineer

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/Jeffallan/claude-skills
cp -r claude-skills/skills/spark-engineer ~/.claude/skills/spark-engineer

Safety box score

Not rated yet. A safety box score grades what a skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. Anyone can request one from this page; it is saved for everyone. How the score works.

Source

Repository

Jeffallan/claude-skills (all skills from this repository)

Path

skills/spark-engineer/SKILL.md

Branch

main

Updated

2026-09-19

Related skills

• aws-billing-and-cost-management — Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer.
• aws-messaging-and-streaming — Guides general use of AWS messaging and streaming services.
• aws-observability — Builds, configures, debugs, and optimizes AWS observability with CloudWatch (Log Insights, Metrics, Alarms, Dashboards, EMF), X-Ray, CloudTrail.
• pandas-pro — Performs pandas DataFrame operations for data analysis, manipulation, and transformation.
• agents-optimize — Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization.
• analytics — When the user wants to set up, improve, or audit analytics tracking and measurement.