ingesting-into-data-lake

An agent skill by aws, from aws/agent-toolkit-for-aws. Tags: data, database, etl, migration, s3.

What it does

Import data into the AWS data lake from S3 files, local uploads, JDBC databases (Oracle, SQL Server, PostgreSQL, MySQL, RDS, Aurora), Amazon Redshift, Snowflake, BigQuery, DynamoDB, or existing Glue catalog tables (migration). Default target is S3 Tables; standard Iceberg on a general purpose bucket is supported where S3 Tables is not adopted. Handles one-time loads, recurring pipelines, migrations. Triggers on: import data, load data, ingest, sync database, migrate table, move data to AWS, set up pipeline, ETL, pull from Snowflake, query BigQuery into S3, export DynamoDB, CTAS, convert to Iceberg. Do NOT use for setting up or troubleshooting Glue connections (use connecting-to-data-source), creating empty tables (use creating-data-lake-table), running queries (use querying-data-lake), finding tables by fuzzy name (use finding-data-lake-assets), catalog audit (use exploring-data-catalog), or SaaS platforms like Salesforce, ServiceNow, SAP, MongoDB, Kafka.

Install

With the skills CLI, which installs into Claude Code, Codex, Cursor and other agents:

npx skills add aws/agent-toolkit-for-aws --skill ingesting-into-data-lake

Or copy the skill folder into Claude Code's skills directory by hand (~/.claude/skills for every project, or .claude/skills inside one):

git clone --depth 1 https://github.com/aws/agent-toolkit-for-aws
cp -r agent-toolkit-for-aws/plugins/aws-data-analytics/skills/ingesting-into-data-lake ~/.claude/skills/ingesting-into-data-lake

Safety box score

Grade D, elevated risk: 51 of 100 risk points, rated 2026-10-21. The grade rates what the skill and its scripts can reach on the machine of whoever installs it, across eight categories from shell execution to secrets access. How the score works.

This skill orchestrates AWS data-lake ingestion and migration workflows across S3, Glue, Athena, DynamoDB, JDBC sources, Snowflake, and BigQuery. Its main risks are broad cloud-side data movement plus creation, overwrite, and deletion of AWS resources and tables, with some reliance on credentials and one unread reference file.

Category levels, from 0 (none) to 3 (high)
Category	Level	Why
Code & shell execution	2 (moderate)	It directs the agent to run AWS CLI/Athena/Glue commands and execute PySpark job templates, e.g. `aws glue create-job`, `aws athena start-query-execution`, and `pip install pyspark boto3`.
Network access	3 (high)	The skill moves data across networks to and from AWS, JDBC databases, Snowflake, BigQuery, and DynamoDB, using commands like `aws s3 cp`, `connection_type="bigquery"`, and `connection_type="snowflake"`.
Filesystem reach	1 (low)	Local-file handling is limited to user-specified paths and uploads (`head`, `file --mime-encoding`, `gzip file.csv`), with no broad system-path access or recursive local deletion.
Secrets & credentials	1 (low)	It touches auth state and secret-bearing services indirectly (`aws sts get-caller-identity`, BigQuery credentials in Secrets Manager), but does not ask for pasted secrets or read key files directly.
Privilege & persistence	2 (moderate)	It creates and updates cloud resources and permissions-sensitive objects such as Glue jobs/crawlers, CloudWatch alarms, and S3 Tables namespaces (`aws glue create-job`, `aws glue create-crawler`, `aws cloudwatch put-metric-alarm`, `aws s3tables create-namespace`).
Instruction hijack surface	0 (none)	No prompt-injection or instruction-evasion content was found; the strong wording is operational guidance, not an attempt to override safeguards.
Irreversible actions	3 (high)	The workflow can create, overwrite, schedule, and delete data-pipeline resources and tables, including `DROP TABLE IF EXISTS temp_source_<timestamp>;`, `aws glue delete-crawler`, `aws glue delete-database`, and recurring pipeline setup.
Transparency	1 (low)	One referenced file could not be read (`references/testing-and-scheduling.md`, size or budget), so part of the workflow could not be inspected.

Before installing

Review `references/glue-job-config.md` and `references/iceberg-catalog-config-and-usage.md` to confirm the job and catalog changes it will make.
Verify the AWS role you will use already has the S3, Glue, Athena, and S3 Tables permissions this skill assumes.
Decide whether you are comfortable with temporary resource cleanup like `aws glue delete-crawler` and `DROP TABLE IF EXISTS temp_source_<timestamp>;`.
Read `references/testing-and-scheduling.md` separately if scheduling/recurrence matters, since it was not inspected here.

Rated 2026-10-21 by an automated review of 25 files from the skill's repository.

Source

Repository: aws/agent-toolkit-for-aws (all skills from this repository)
Path: plugins/aws-data-analytics/skills/ingesting-into-data-lake/SKILL.md
Branch: main
Collection: aws-data-analytics
Updated: 2026-09-19

exporting-rds-to-s3 — Exports Amazon RDS or Aurora database snapshots to Amazon S3 in Apache Parquet format for analytics, backup, or data migration.
creating-data-lake-table — Create managed Iceberg tables using Amazon S3 Tables (s3tables API namespace) with automatic compaction and snapshot management.
finding-data-lake-assets — Resolve data lake and lakehouse asset references across Glue Data Catalog, S3, S3 Tables, and Redshift.
principle-model-the-domain — Apply when writing stateful logic, or when code branches a lot or repeats a shape assumption across files.
amazon-documentdb — Manages Amazon DocumentDB end-to-end — serverless-on-8.0 cluster setup, TLS/VPC/driver config, flexible-schema and vector-search data modeling.
amazon-keyspaces — Provides authoritative compatibility checks, pricing estimates, connection troubleshooting, pre-warming guidance.