Azure Databricks Developer

Role Overview

Seeking an experienced Azure Databricks Developer to design, develop, integrate, and support cloud-native data engineering solutions and enterprise applications using Microsoft Azure technologies. The role is responsible for analyzing business and technical requirements, designing scalable ETL/ELT solutions, migrating legacy Informatica PowerCenter workloads to Azure Databricks, and participating throughout the Software Development Life Cycle (SDLC).

Responsibilities

• Analyze business, functional, and technical requirements and recommend effective application and data engineering solutions.
• Design, develop, and maintain cloud-native ETL/ELT pipelines using Azure Databricks, PySpark, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Lake Storage Gen2 (ADLS Gen2), Python, and SQL.
• Assess and migrate Informatica PowerCenter workflows, mappings, and ETL processes to Azure Databricks.
• Participate in all phases of the SDLC, including requirements analysis, conceptual design, system design, software integration, testing, deployment, implementation, maintenance, and lifecycle support.
• Design scalable batch, incremental, and Change Data Capture (CDC) data pipelines.
• Integrate Azure data platforms with SQL Server, Oracle, REST APIs, Power BI, and other enterprise applications.
• Optimize Spark jobs, ETL processes, and Azure workloads for performance, scalability, and reliability.
• Perform software integration, configuration management, release management, and deployment activities.
• Conduct unit, system, integration, and user acceptance testing; analyze software test results and recommend corrective actions.
• Present system designs, technical architecture, and implementation strategies during formal design and user review sessions.
• Implement CI/CD pipelines using Azure DevOps and Git.
• Implement Azure security best practices using Azure Key Vault, Microsoft Entra ID, RBAC, and managed identities.
• Develop technical documentation, architecture diagrams, operational procedures, and knowledge transfer materials.
• Provide production support, troubleshooting, application maintenance, monitoring, and lifecycle management for enterprise data platforms.

Requirements

• Bachelor's Degree in Computer Science, Information Systems, Engineering, or a related technical discipline.
• Minimum 3 years of professional experience in Azure Data Engineering, Software Engineering, or Application Development.
• Experience implementing enterprise software solutions using structured SDLC methodologies.
• Strong expertise in Azure Databricks, PySpark, Spark SQL, Python, Azure Data Factory, Azure Synapse Analytics, ADLS Gen2, and Delta Lake.
• Experience migrating Informatica PowerCenter workloads to Azure Databricks.
• Strong SQL/T-SQL skills with experience integrating SQL Server, Oracle, REST APIs, and Power BI.
• Experience with Azure DevOps, Git, CI/CD, configuration management, and software integration.
• Experience performing requirements analysis, system design, software testing, deployment, and lifecycle support.
• Ability to provide solutions for moderate-complexity software problems and recommend corrective actions based on testing results.
• Experience presenting technical designs during formal design reviews.