Built a Dockerized ETL pipeline that downloads NYC taxi data from a public URL and ingests it into a PostgreSQL database. Automated with Python scripts and optimized for scalability.
Tech Stack: Python, Docker, PostgreSQL
View on GitHubAn end-to-end data pipeline for processing, transforming, and visualizing stock data for Facebook, Apple, Amazon, Netflix, and Google.
Tech Stack: SQL, BigQuery, dbt
View on GitHubComing soon: A real-time pipeline that ingests weather data from APIs, processes it with pandas, and stores it in a cloud database.
Tech Stack: Python, Airflow, PostgreSQL, AWS S3
In Progress