Role Overview
We are seeking a Data Engineer with GCP to design, develop, and maintain scalable data pipelines on Google Cloud. You will build and optimise ETL/ELT workflows for batch and real-time data processing, work with BigQuery for data warehousing and SQL development, and ensure data quality, security, and governance across the platform.
Responsibilities
- Design, develop, and maintain scalable data pipelines using GCP services.
- Build and optimise ETL/ELT workflows for batch and real-time data processing.
- Work with Google BigQuery for data warehousing, SQL development, and query optimisation.
- Develop data processing solutions using Python, PySpark, and Apache Beam.
- Implement data integration using Cloud Dataflow, Cloud Composer, and Pub/Sub.
- Build and manage data lakes and data warehouses on Google Cloud.
- Implement data quality checks, data validation, and monitoring.
- Optimise pipeline performance, scalability, and cost efficiency.
- Collaborate with data analysts, data scientists, architects, and business teams.
- Ensure data security, governance, and compliance best practices.
Requirements
- Strong experience in Google Cloud Platform (GCP).
- Hands-on experience with BigQuery.
- Strong programming skills in Python and SQL.
- Experience developing ETL/ELT data pipelines.
- Experience with Cloud Dataflow and/or Apache Beam.
- Knowledge of Cloud Storage (GCS).
- Experience with data modelling and data warehousing.
- Experience with batch and real-time data processing.