Role Overview
As a Databricks Platform Engineer you will design and build scalable cloud-based data platforms supporting advanced analytics AI and data engineering workloads. Collaborate with data engineers architects and business stakeholders to enable secure high-performing Databricks environments. Hybrid role based in Canberra - Ngambri with flexible reduced-hours options.
Responsibilities
- Design build and maintain scalable Databricks-based data platforms across Azure or AWS.
- Configure and manage Databricks workspaces clusters jobs workflows and permissions for data engineering analytics and AI use cases.
- Support Lakehouse architectures using Databricks Apache Spark Delta Lake Unity Catalog and related cloud-native services.
- Develop and maintain infrastructure-as-code CI/CD pipelines and automation for repeatable platform delivery.
- Implement platform security access controls governance monitoring and operational standards across cloud and Databricks environments.
- Work with data engineers AI engineers and architects to design platform patterns for ingestion transformation orchestration storage and consumption.
- Troubleshoot platform issues across Databricks Spark and cloud services resolving incidents and improving stability.
- Document platform designs configuration patterns operational processes and technical standards.
- Support and guide team members contributing to delivery planning and technical direction.
Requirements
- Strong experience designing and implementing cloud-based data platforms Azure and/or AWS.
- Hands-on experience working with the Databricks Data Intelligence Platform including workspace configuration compute management jobs workflows and platform administration.
- Proficiency in Databricks Apache Spark Delta Lake Unity Catalog and related data engineering technologies.
- Experience supporting secure and governed data platform environments including identity access management permissions secrets networking and data governance controls.
- Knowledge of infrastructure-as-code and DevOps practices e.g. Terraform CI/CD pipelines Git-based workflows and automated deployment processes.
- Strong problem-solving and analytical capability with a practical approach to technical issues.
- Excellent communication skills with the ability to engage technical and non-technical stakeholders.
- Experience contributing to or leading delivery within a team environment including working with engineers architects and stakeholders to achieve shared outcomes.