01 / Data Engineering
RetailPulse Data Platform
Built retail ETL/ELT pipelines integrating 6 PostgreSQL source tables through S3 Bronze/Silver layers into Snowflake dimensional models.
- Implemented watermark-based incremental extraction with 5,000-row fetch batches and watermark updates after successful S3 uploads.
- Orchestrated a 10-task Airflow DAG; demonstrated 6 concurrent Glue runs completing in 67–88 seconds with zero retries.
- Developed 6 Gold marts, 43 dbt data tests, and 27 Glue validation checks; verified 10,040 Silver customer records with Athena.
APACHE AIRFLOW · 10-TASK DAG
6 Gold marts · 43 dbt tests
Illustrative UI