Role Overview
We are looking for a seasoned Data Engineer passionate about building cutting-edge cloud data solutions. You will design, optimize, and scale cloud-based data architecture and high-performance ETL/ELT pipelines using a modern stack including Azure, PySpark, and Microsoft Fabric.
Responsibilities
- Design, build, and maintain robust end-to-end ETL/ELT pipelines for large-scale datasets.
- Develop data workflows using Microsoft Fabric Data Factory, Notebooks, and PySpark.
- Manage Lakehouse and OneLake centralized storage to power enterprise analytics.
- Migrate legacy databases to Azure SQL/Synapse SQL and optimize performance.
- Automate data workflows using Azure Functions and ADF triggers.
- Build and refine SQL Server objects including stored procedures, views, and triggers.
- Provide operational support across various shifts and on-call rotations.
- Collaborate with Data Scientists and Business Analysts to deliver production-ready datasets.
Requirements
- 5+ years of experience in cloud data engineering and big data processing.
- Deep hands-on experience with Azure Databricks, Azure Synapse Analytics, and ADLS.
- 6+ months of hands-on experience with Microsoft Fabric Pipelines, Data Factory, and Notebooks.
- Advanced proficiency in PySpark, Delta Lake, and expert-level SQL.
- Bachelor’s or Master’s degree in CS, IT, or a related field.
Skills
- PySpark
- Azure Databricks
- Microsoft Fabric
- SQL
- Azure Synapse Analytics
Nice to Have
- Microsoft Certifications (e.g., DP-700).
- MBA or equivalent technical degree.