Junior Data Engineer (3y+ Exp)

Role Overview

We are looking for a skilled Data Engineer with 3–5 years of experience in building scalable data pipelines and integration solutions. The ideal candidate should have strong hands-on expertise in Databricks, Python, and API-based data integration.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Python.
• Build and optimize data workflows using Apache Spark within the Databricks environment.
• Integrate data from internal systems and third-party platforms through REST APIs and other integration mechanisms.
• Develop reusable data ingestion frameworks and automation scripts.
• Perform data transformation, cleansing, validation, and enrichment for analytics and reporting.
• Work with structured and semi-structured data sources such as JSON, CSV, APIs, databases, and cloud storage.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Collaborate with Data Analysts, Data Scientists, and application teams to deliver reliable datasets.
• Implement monitoring, logging, and error-handling mechanisms for production pipelines.
• Participate in code reviews, testing, and deployment activities following engineering best practices.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• 3–5 years of experience in Data Engineering or Big Data development.
• Strong hands-on experience with Databricks (mandatory).
• Proficiency in Python for data processing and automation.
• Experience with REST APIs, API authentication, and data integration.
• Strong understanding of Apache Spark and distributed data processing.
• Experience writing complex SQL queries and working with relational databases.
• Knowledge of data lake/lakehouse concepts and Delta Lake.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Familiarity with Git, CI/CD pipelines, and Agile development practices.

Nice to Have

• Experience with Azure Databricks and Azure Data Factory.
• Knowledge of orchestration tools such as Airflow or Databricks Workflows.
• Exposure to streaming technologies such as Kafka.
• Understanding of data governance, security, and access control.

Benefits

• Work from home

Skills

• Databricks
• Python
• Apache Spark
• REST APIs
• SQL