Data Engineer MS fabric

Role Overview

We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines within the Microsoft Fabric ecosystem. You will be responsible for managing the end-to-end data lifecycle, from ingestion into bronze layers to the development of gold layer models for advanced analytics and reporting.

Responsibilities

• Perform data analysis to understand source systems and business requirements.
• Translate requirements into scalable data models and pipeline designs.
• Build and maintain data ingestion pipelines (batch and incremental/delta loads) into Microsoft Fabric.
• Ingest and land raw data into bronze, silver, and gold layers using ETL/ELT methodologies.
• Implement data quality rules, validation checks, and robust error handling mechanisms.
• Design self-healing pipelines with auto-retry, checkpointing, and idempotent processing.
• Optimize pipeline performance and proactively monitor for bottlenecks.
• Collaborate with CI/CD and DevOps practices for multi-environment deployment.
• Work closely with business, BI, and architecture teams to deliver end-to-end solutions.

Requirements

• Proven experience in designing end-to-end ETL/ELT pipelines.
• Strong ability to write, amend, and debug scripts using PySpark and SQL.
• Experience working with database tables, flat files, and APIs.
• Knowledge of software design standards and data modeling (fact/dimension tables).
• Familiarity with DevOps and CI/CD workflows.

Skills

• Microsoft Fabric
• PySpark
• SQL
• ETL/ELT
• Data Modeling