Databricks Data Engineering Consultant

Location: Sydney or Melbourne (Hybrid)

Contract position (8 weeks)

Our client is seeking an experienced Databricks Data Engineer for an 8-week contract to bring the Databricks Proof of Technology (PoT) to completion and stand up a simulated MVP environment. This role supports the Global Optimisation Platform Project, which is creating an creative optimisation and bidding solution.

About the role:

- Be part of a core Data Engineering team, this engagement adds the specialist Databricks capacity to close that gap across the team.
- Contributing direct Databricks engineering effort within the existing team, with the goal of clearing the PoT gate on schedule and handing Quant Research a workable simulated MVP.
- Stand up the core Databricks environment for the simulated MVP, with security built in through Unity Catalog, access controls, and compute policies.
- Create simulated asset telemetry and align the market data, giving the Quant Research team a complete, end-to-end modelling pipeline
- Use real effort from reference pipelines as a baseline to refine the delivery-phase estimate for the data ingestion and integration workstream.





Required Experience:

- Over five years of data engineering experience, including a minimum of two years building and running Azure Databricks platforms in production.
- Hands-on experience designing and governing Unity Catalog, covering catalog structure, access permissions, external locations, storage credentials and system tables.
- Deep knowledge of Azure Databricks security, including VNet injection, Private Link, Entra ID with SCIM provisioning, service principals and Key Vault-backed secret scopes.
- Proven experience building real-time pipelines with Structured Streaming on Event Hubs or Kafka, along with Delta Lake and medallion architecture.
- Practical experience deploying with Databricks Asset Bundles and running CI/CD pipelines in Azure DevOps or GitHub, plus working with Git folders.
- Advanced Python skills, including building wheel packages and managing library dependencies in privately networked environments.

Should you have the experience and skills required to be a success in this role, hit APPLY today!

📌 Data Engineer (Southern Sydney)
🏢 Source Point Strategic Sourcing
📍 Southern Sydney