Lead Data Engineer (Databricks) (Brisbane)

Lead Data Engineer (Databricks) (Brisbane)

19 Nov
|
CareCone Group
|
Brisbane

19 Nov

CareCone Group

Brisbane

Job Title: Lead Data Engineer (Databricks)

Experience: 10+ Years

Location: Brisbane (On-site)

Job Summary

We are seeking an experienced Databricks Lead to drive the design, development, modernization, and operational excellence of enterprise-scale data platforms on Databricks and AWS . The ideal candidate will combine deep technical expertise with team leadership, stakeholder management, and platform governance capabilities.

As a Databricks Lead, you will be responsible for leading a team of data engineers, defining best practices, overseeing delivery, ensuring platform stability, and managing BAU operations while partnering with business and technology stakeholders to deliver scalable, high-performing data solutions.

The role requires solid experience across Databricks Lakehouse Architecture, PySpark, Data Engineering, AWS, DataOps, and ITIL-based support operations .

Technical Competencies

Databricks & AWS Platform Leadership

- Extensive hands-on experience with Databricks Lakehouse Platform and enterprise-scale implementations.
- Strong expertise in:
- PySpark
- Delta Lake
- Unity Catalog
- Delta Live Tables (DLT) / Lakeflow Pipelines
- Serverless Compute
- SQL Warehouses




- Databricks Dashboards & Alerting
- Lead data architecture discussions and establish platform standards and best practices.
- Strong experience managing Databricks environments, security models, and governance frameworks.

Performance Optimization

- Expertise in:
- Liquid Clustering
- Z-Ordering
- Spark Performance Tuning
- Query Optimization
- Cost Optimization and Workload Management
- Experience optimizing large-scale batch and real-time processing workloads.

AWS Cloud

- Strong experience with AWS services including:
- Amazon S3
- IAM
- Lambda
- CloudWatch
- SNS/SQS
- Secrets Manager
- Experience designing secure, scalable, and highly available cloud-based data platforms.

Data Engineering & Integration

- Advanced experience with:
- Databricks Asset Bundles (DABs)
- dbt Core / dbt Cloud
- Kafka Streaming
- Structured Streaming
- Data Modeling
- ETL/ELT Frameworks
- Experience handling large-scale structured and semi-structured datasets.

Interested Candidates can share their updated resumes to [email protected] or can reach me at +61 291 575 539

📌 Lead Data Engineer (Databricks) (Brisbane)
🏢 CareCone Group
📍 Brisbane

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: lead data engineer (databricks) (brisbane) / brisbane

Subscribe to this job alert:

Get the latest job offers by email for: lead data engineer (databricks) (brisbane) / brisbane