Data Pipelines

Data Pipelines taxonomy generated by the site skill importer.

3 skills
W
spark-optimization

by wshobson

spark-optimization is a practical guide to diagnosing slow Apache Spark jobs with partitioning, shuffle, skew, caching, and memory tuning. Use it to install the skill from wshobson/agents, read SKILL.md, and apply evidence-based fixes from Spark UI symptoms, cluster settings, and query patterns.

Performance Optimization
Favorites 0GitHub 32.6k
W
dbt-transformation-patterns

by wshobson

dbt-transformation-patterns helps agents structure dbt projects with staging, intermediate, and marts layers, plus testing, documentation, and incremental model guidance. Use it to plan installs, scaffold new repos, or refactor SQL into cleaner analytics engineering patterns for Database Engineering teams.

Database Engineering
Favorites 0GitHub 32.6k
W
airflow-dag-patterns

by wshobson

airflow-dag-patterns helps design production-ready Apache Airflow DAGs with stronger task patterns, dependencies, operators, sensors, testing, and deployment guidance for scheduled jobs.

Scheduled Jobs
Favorites 0GitHub 32.6k