Download App
Data Engineer
WebSenor InfoTech
Actively HiringFull-time
Location
Remote, IN
Experience
2-4 years
Role
Data Engineer
Salary
Competitive
Your Application Toolkit
Stand out. Get hired.
80% of applicants fail the initial ATS screen. Check your compatibility before you apply and stand out from the crowd.
Job Description
Role Overview
We are seeking a highly skilled Data Engineer with expertise in Azure Cloud and modern data engineering technologies to build scalable, reliable, and high-performance data platforms. You will be responsible for designing, developing, and maintaining enterprise-grade data pipelines, enabling advanced analytics, machine learning, and business intelligence solutions across the organization.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
- Build and optimize ETL/ELT workflows using Azure Data Factory (ADF).
- Implement and manage Delta Lake architecture to ensure reliable, scalable, and ACID-compliant data processing.
- Develop orchestration workflows using Apache Airflow and Azure Logic Apps.
- Manage structured and unstructured data in Azure Data Lake Storage Gen2 (ADLS Gen2).
- Develop efficient Python applications and optimized SQL queries for data processing and analytics.
- Build and maintain CI/CD pipelines using Azure DevOps or GitHub Actions to automate deployment and testing.
- Implement MLflow for experiment tracking and machine learning lifecycle management.
- Monitor data pipelines using Data Observability tools to ensure data quality and performance.
Requirements
- Strong hands-on experience with Azure Databricks and PySpark.
- Experience implementing Delta Lake architecture.
- Proficiency in Azure Data Factory (ADF).
- Experience with Apache Airflow and Azure Logic Apps.
- Strong programming skills in Python and advanced knowledge of SQL.
- Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Experience implementing CI/CD pipelines.
- Hands-on experience with MLflow and Data Observability tools.
- Strong understanding of distributed computing and Spark optimization.
- Experience working in Microsoft Azure Cloud environments.
Skills
- Azure Databricks
- PySpark
- Python
- Azure Data Factory
- SQL
Preferred Qualifications
- Bachelor's or Master's degree in Computer Science or a related field.
- Microsoft Azure certifications such as Azure Data Engineer Associate.
- Experience with Azure Synapse Analytics, Microsoft Fabric, or Power BI.
- Knowledge of data warehousing and dimensional modeling.
Required Skills
Azure DatabricksPySparkPythonAzure Data FactorySQL
Ready to make your move?
Take the next step in your career. Apply directly on the company's platform.
Apply for this role