Accenture Australia logo for Data Engineer – AWS Kinesis & Databricks - AU Citizen job on DevsUnite

Data Engineer – AWS Kinesis & Databricks - AU Citizen

Accenture Australia

Actively HiringFull-time
Apply
Location

Canberra, Australian Capital Territory, Australia

Experience

2-4 years

Role

Data Engineer – AWS Kinesis & Databricks - AU Citizen

Salary

Competitive

Share this job

Know someone who'd be a great fit?

Job Description

Role Overview

We’re looking for a hands-on Data Engineer with strong experience building reliable, real-time data pipelines using Amazon Kinesis and Databricks. You’ll work with modern AWS data technologies to develop and optimise streaming pipelines, moving high-volume data from Amazon Kinesis into the Databricks Lakehouse using Spark Structured Streaming and Delta tables.

Responsibilities

  • Build and support real-time data pipelines using Amazon Kinesis Data Streams and Kinesis Data Firehose.
  • Develop streaming solutions in Databricks using Spark Structured Streaming.
  • Ingest and manage streaming data within Delta tables.
  • Optimise pipelines for performance, latency, reliability and cost.
  • Build monitoring and alerting using AWS CloudWatch.
  • Work with batch data engineers to ensure streaming and batch pipelines integrate effectively.
  • Develop clean, testable Python and SQL code.
  • Troubleshoot pipeline and data-processing issues and contribute to continuous improvement.

Requirements

  • Hands-on production experience with Amazon Kinesis Data Streams and/or Kinesis Data Firehose. Kinesis experience is essential.
  • Strong Python and SQL skills.
  • Experience with Apache Spark Structured Streaming.
  • Hands-on experience with Databricks, ideally including Delta Lake and Unity Catalog.
  • Good knowledge of AWS services, particularly IAM, S3 and CloudWatch.
  • Understanding of scalable and reliable data-pipeline design.
  • Strong troubleshooting and problem-solving skills.
  • Experience with Kafka is beneficial

Benefits

  • 18 weeks paid parental leave
  • Long & short-term career break opportunities
  • Structured career development program
  • Local and international career opportunities.
  • Certified as a Family Inclusive Workplace™
  • Flexible Work Arrangements – centred around Accenture’s Truly Human ethos and our commitment to supporting the health and wellbeing of our people.
  • We are proud to be in the top 3 of last year’s Diversity & Inclusion Index!

Required Skills

Amazon KinesisDatabricksSpark Structured StreamingPythonSQL

Ready to apply for Data Engineer – AWS Kinesis & Databricks - AU Citizen?

See how you stack up at Accenture Australia before you apply, or go straight to the application.

Apply Anyway