Data Integration Engineer - Remote

Role Overview

We are looking for a skilled Data Integration Engineer (full-time or part-time) with hands-on experience in Databricks to design, develop, and maintain scalable data integration and data processing solutions. This is a remote position requiring 2–6 years of experience.

Responsibilities

• Design and develop scalable data integration and ETL/ELT pipelines using Databricks.
• Develop and optimize data processing workflows using Databricks, Apache Spark, and PySpark.
• Integrate data from multiple sources such as databases, APIs, files, and cloud platforms.
• Build and maintain reliable data pipelines for batch and near-real-time processing.
• Work with Delta Lake for data storage, transformation, and optimization.
• Develop complex SQL queries for data transformation and analysis.
• Implement data validation, quality checks, and error-handling mechanisms.
• Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
• Collaborate with technical teams to understand data requirements.
• Monitor, troubleshoot, and resolve data pipeline and integration issues.
• Maintain technical documentation for data pipelines and integration processes.

Requirements

• 2–6 years of experience in Databricks and data integration.
• Hands-on experience with Databricks and Apache Spark / PySpark.
• Strong SQL skills.
• Experience in ETL/ELT and data integration.
• Good understanding of Delta Lake and Databricks data processing.
• Experience with Python scripting.
• Knowledge of data warehousing concepts and data modeling.
• Experience working with REST APIs, JSON, and CSV.
• Understanding of cloud data platforms such as Azure, AWS, or GCP.

Skills

• Databricks
• Apache Spark
• PySpark
• SQL
• Python

Compensation

Pay: ₹500,000.00 - ₹1,300,000.00 per year