Role Overview
We are seeking a highly skilled and experienced Cloud Data Engineer to join our team for a Cloud Data Modernization project. The role involves migrating ETL workloads from on-premises platforms to Azure Cloud, Databricks, and Snowflake.
Responsibilities
- Contribute to the migration of the ETL workloads from on-premises platforms to Azure Cloud, Databricks and Snowflake.
- Develop and implement data platform solutions using Databricks, Azure Data Factory (ADF), Self-hosted Integration Runtime (SHIR), Logic Apps, Azure Data Lake Storage Gen2 (ADLS Gen2), Blob Storage, and Snowflake.
- Review and analyze existing on-premises ETL processes.
- Implement DevOps practices and CI/CD pipelines using GitHub Actions.
- Collaborate with cross-functional teams to ensure seamless integration and data flow.
- Tune and optimize workloads for performance scalability.
- Implement data quality validation, monitoring, and observability processes.
- Support the development and maintenance of automated reconciliation and data validation processes.
- Ensure data security and compliance with industry standards.
Requirements
- Minimum of 3+ years of experience as a Cloud Data Engineer.
- Hands-on experience with Databricks, Azure Cloud data tools (ADF, SHIR, Logic Apps, ADLS Gen2, Blob Storage), Apache Spark, PySpark and Snowflake.
- Experience developing or supporting data migration and cloud modernization initiatives.
- Strong experience in ETL development using on-premises databases and ETL technologies.
- Experience with Python or other scripting languages for data processing.
- Strong SQL development experience including complex queries, stored procedures, views, and ETL transformations.
- Experience with GitHub branching, pull requests, and GitHub Actions.
- Experience with code reviews and engineering best practices.
- Experience supporting automated testing and data quality processes.
- Experience working within Agile delivery methodologies.
- Experience using AI-assisted development tools (GitHub Copilot, Databricks Assistant, ChatGPT, Claude, etc.).
Skills
- Databricks
- Snowflake
- Azure
- Python
- SQL