Full‑time
Data Engineer — Quantiphi
AWS PySpark Redshift Snowflake Python SQL LLM
Impact
- Migrated 78+ Statistical Analysis System (SAS) processes to PySpark on AWS Glue, reducing execution time by ~30% while preserving functional parity.
- Worked across three client groups; led Business Acceptance Testing (BAT) and End‑User Testing (EUT), partnering with business users to validate outputs against legacy SAS while maintaining compliance and data integrity.
- Engineered a 50+ TB financial data warehouse in AWS Redshift; designed ETL pipelines and optimized SQL to support production revenue planning/reporting.
- Automated ingestion + file transfer pipelines with Python, reducing manual upload/validation work by 70%.
- Built an automated Redshift vacuum trigger using AWS Step Functions and ECS, reducing downtime by 40%.
- Performed EDA on 10+ TB financial datasets and improved data quality by 30% through targeted fixes and validation checks.
- Developed an LLM utility for an AI-powered agent, reducing manual efforts for project documentation by 50%.
- Designed and optimized SQL workflows in Snowflake to improve query performance for analytics and reporting workloads.
- Received recognition in a client newsletter for independently driving test cycles and delivery coordination.