본문으로 건너뛰기
TakoAPI
스킬로 돌아가기

senior-data-engineer

alirezarezvani 제작

Data & Analytics
1.8kclawskills.sh가 보고한 다운로드 수
2스타
1조회수
좋아요 0개로그인하면 이 스킬에 좋아요를 누를 수 있습니다
—아직 평가 없음
내 평가

요약

Data engineering skill for building scalable.

설명

이 스킬이 하는 일

Covers production-grade data pipeline design, ETL/ELT implementation, and data infrastructure decisions. Handles batch and streaming architectures using Python, SQL, Spark, Airflow, dbt, and Kafka. Includes data modeling, quality validation, and DataOps practices.

Combines architecture decision frameworks with ready-to-run scripts for orchestration, quality validation, and performance analysis, covering the full modern data stack in one place.

사용 시점

  • Building a daily batch ETL from Postgres to Snowflake
  • Setting up Airflow DAGs for scheduled data ingestion
  • Validating data quality on freshness, completeness, and uniqueness
  • Choosing between Lambda and Kappa pipeline architectures
  • Optimizing a slow Spark job processing large parquet files

예시 워크플로

입력

User asks: Design a pipeline to ingest sales data from Postgres into Snowflake daily

에이전트
  1. 1

    Evaluates latency and volume requirements to confirm batch processing is appropriate

  2. 2

    Generates an Airflow DAG config with a scheduled daily run targeting Snowflake

  3. 3

    Defines dbt models for transformation and dimensional modeling in the warehouse

  4. 4

    Adds data quality checks for freshness, completeness, and uniqueness on the sales dataset

출력

Airflow DAG, dbt model files, and a data quality validation script ready for deployment

설치

clawhub install alirezarezvani/senior-data-engineer

CLI 명령을 실행하거나, 프롬프트를 Claude Code, Cursor, Codex 등 사용하는 코딩 에이전트에 붙여넣으세요.

링크