21 Oct
|
Atturra
|
Melbourne
What You’ll Be Doing
Atturra Data & Integration is seeking an experienced Senior Databricks Data Architect to join our growing team on a permanent basis.
As a Senior Databricks Data Architect, you will define and lead the technical uplift of the Databricks Lakehouse Platform for our clients, ensuring the foundations are in place for governed, automated, and production-grade data delivery.
Your primary engagement will be a financial services member organisation Data Strategy Activation program (September–December 2026), where you will lead Item 2.1 — Platform and Governance Foundation.
This is the prerequisite scope item for everything else in the program: no governed pipelines, no RBAC, no environment separation, and no downstream tooling can operate without the work you will design and oversee in this item. You will carry 31 architect days across September and October, with the item running to handover by end of October.
What You’ll Deliver On The Client Engagement:
- Unity Catalog metastore provisioning and catalog hierarchy design — establishing the single metastore and catalog structure that all governed pipelines, workspace binding, and access controls depend on.
- Purview architecture, classification and PII handling design — defining classification taxonomy and PII controls before Silver/Gold data is used, meeting the roadmap's requirement for governed access and masking controls.
- CI/CD and environment architecture — designing the dev/test/prod promotion model and the target environment separation structure that Azure DevOps and Databricks Asset Bundles must implement.
- Lakeflow Declarative Pipelines design and ADF migration architecture — defining the Lakeflow ingestion pattern and migration path from Azure Data Factory, resolving the dual-pipeline/single-cluster design conflict identified in the data strategy.
Alongside the architecture work, you will provide technical governance throughout delivery — reviewing and approving deliverables produced by the Senior Data Engineer as they build out CI/CD automation, Purview deployment, Unity Catalog binding, source scanning, and cluster log streaming.
Broader Role Responsibilities:
- Translating data strategy and roadmap requirements into technically sound platform designs that engineering teams can build and clients can operate independently after handover.
- Defining ingestion patterns, environment structures, security models, and observability approaches at the start of each engagement so that all build activity is anchored to an agreed architecture.
- Reviewing key technical deliverables across the engagement lifecycle, flagging design drift, and resolving blockers that require architectural decisions.
- Collaborating with client stakeholders — including technical leads and data owners — to validate architecture decisions and obtain sign‑off before build commences.
- Contributing to presales and scoping activities within the Data Platforms practice, including effort estimation, technical responses, and solution design.
Working Conditions:
- Open to Permanent Residency Visa
- Permanent full time or Contract
- Hybrid
- Open to all major locations e.g., Sydney/Canberra/Melbourne/Brisbane/Perth
- Occasional travel to client site/s
What will make you successful in this role?
Essential
- At least 5 years of data platform experience, with at least 2 years in a hands‑on architecture role on Databricks — not purely advisory.
- Current Databricks certification(s) required (e.g., Databricks Certified Data Engineer Professional or Databricks Certified Associate Developer for Apache Spark).
- Deep hands‑on experience with Unity Catalog — including metastore provisioning, catalog and schema hierarchy design, workspace binding, and access model implementation.
- Demonstrated experience designing and implementing CI/CD pipelines for Databricks using Azure DevOps and Databricks Asset Bundles, including dev/test/prod environment separation.
- Understanding of Infrastructure as Code principles and tooling (e.g., Terraform, ARM templates, or Bicep) for provisioning and managing Azure data platform resources in a repeatable, auditable way.
- Experience designing and deploying Microsoft Purview as an enterprise governance platform, including classification configuration, PII handling controls, lineage scanning, and source registration.
- Experience with Lakeflow Declarative Pipelines (or Databricks Delta Live Tables) and the ability to design ingestion patterns and migration paths from Azure Data Factory.
- Robust understanding of medallion architecture (Bronze/Silver/Gold) and the ability to define standards and patterns that engineering teams build to consistently.
- Ability to produce c
📌 Senior Data Architect (Melbourne)
🏢 Atturra
📍 Melbourne