Role Overview
We are seeking an experienced Data Engineer to support the development, enhancement, validation, and quality assurance of our AI agents that underpin our digital-first capabilities. This role focuses on ensuring AI Agent workings and outputs are accurate, reliable, and aligned with defined business and technical requirements while maintaining strong data governance and security standards.
Responsibilities
- Build and maintain data pipelines and validation frameworks to assess AI Agent performance and output quality.
- Pass and update context provided to the Agent to improve the efficiency and output of the model and the data pipeline.
- Design automated quality checks, monitoring, and reporting for AI agent behavior across batch and streaming data.
- Analyze AI agent outputs to identify gaps, risks, and improvement opportunities.
- Collaborate with Business Teams to develop solutions.
- Support the definition of AI agent success metrics, acceptance criteria, and enhancement priorities.
- Partner with technology, analytics, and platform teams to improve AI agent reliability, transparency, and scalability.
- Ensure solutions are cost-effective and aligned with data governance, security, and regulatory standards.
Requirements
- Strong experience as an AWS Data Engineer with hands-on programming skills.
- Experience with AWS (S3, Glue, Athena, EMR, Lambda), Snowflake, and data warehousing concepts.
- Strong SQL, Serverless, and Python skills, with experience building ETL/ELT pipelines.
- Understanding of batch and streaming architectures (e.g. Kafka, Airflow).
- Experience implementing data quality, monitoring, site reliability, or validation controls.
- Excellent communication and problem-solving skills.
Skills
- Python
- AWS
- SQL
- Snowflake
- Apache Airflow