Data Engineer

Role Overview

Join Wipro Technologies as a Data Engineer and be a key contributor to transforming our data architecture. You will design, build, and optimize scalable data pipelines leveraging cutting-edge technologies with a strong focus on the Medallion Architecture to ensure efficient and robust data flow throughout the organization.

Responsibilities

• Gather detailed requirements through workshops and client engagement sessions.
• Develop robust, scalable data pipelines and modules that integrate seamlessly across system components.
• Conduct comprehensive testing, trial runs, and participate in User Acceptance Testing (UAT).
• Support smooth implementation and integration of data applications within the enterprise ecosystem.
• Maintain thorough documentation, including flowcharts, system diagrams, and well-structured code comments.
• Collaborate with cross-functional teams including business analysts, architects, and operations.

Requirements

• Hands-on expertise in cloud environments, particularly Azure Data Lake Storage (ADLS) and AWS S3.
• Strong command in distributed data processing and data warehousing concepts.
• Experience implementing modern CI/CD pipelines using Git and Azure DevOps.
• Ability to perform rigorous data quality validation and troubleshoot production issues.
• Experience with Medallion Architecture and optimizing Spark job performance.

Skills

• PySpark
• Databricks
• Delta Lake
• Azure
• AWS

Nice to Have

• Experience with Apache Airflow, Kafka, and Unity Catalog.
• Experience within the BFSI domain.