Role Overview
We are looking for an experienced Data Engineer with strong hands-on expertise in Apache Airflow to design, develop, and maintain reliable data pipelines and workflow orchestration solutions.
Responsibilities
- Design, develop, and maintain Airflow DAGs for data pipeline orchestration.
- Build and optimize scalable ETL/ELT pipelines.
- Monitor, troubleshoot, and optimize Airflow workflows and task performance.
- Implement scheduling, dependencies, retries, alerts, and error handling in DAGs.
- Integrate Airflow with databases, cloud platforms, APIs, and data processing tools.
- Ensure data quality, pipeline reliability, and operational efficiency.
- Collaborate with Data Engineers, Analysts, and other technical teams.
Requirements
- 4+ years of experience in Data Engineering.
- Strong hands-on experience with Apache Airflow and DAG development.
- Proficiency in Python and SQL.
- Experience with ETL/ELT pipelines and data orchestration.
- Understanding of databases, data warehouses, and cloud environments.
- Experience with Git and CI/CD practices.
- Strong debugging and problem-solving skills.
- Tier 1 – Required: Astronomer Certification for Apache Airflow Fundamentals (for DAG Authors).
- Tier 2 – Preferred: Astronomer Certification for DAG Authoring.
Skills
- Apache Airflow
- Python
- SQL
- ETL/ELT
- Git
Good to Have
- Experience with Astronomer or managed Airflow environments.
- Knowledge of AWS/Azure/GCP data services.
- Experience with Spark, dbt, Kafka, or other modern data engineering tools.