Databricks Tech Lead

Location: Australia

Employment Type: Contract / Full-time

We are seeking an experienced Databricks Tech Lead to lead the design, development, and implementation of enterprise-scale data engineering solutions on the Databricks Lakehouse platform. The ideal candidate will have solid expertise in Databricks, Apache Spark, cloud technologies, and data architecture while leading technical teams and driving best practices.

Key Responsibilities

- Lead the design and implementation of scalable data engineering solutions using Databricks.
- Develop and optimize batch and streaming data pipelines. Build ETL/ELT workflows using PySpark, Spark SQL, and Databricks SQL.
- Design and implement Delta Lake and Medallion Architecture (Bronze, Silver, Gold). Collaborate with architects, business stakeholders, and cross-functional teams.
- Mentor data engineers and perform technical reviews. Optimize Spark workloads for performance and cost. Implement CI/CD pipelines and DevOps best practices.
- Ensure data quality, governance, security, and compliance.



Support production deployments and resolve technical issues.

Required Skills

- 8+ years of experience in Data Engineering.
- 3+ years of hands‑on experience with Databricks.
- Strong expertise in Databricks Lakehouse, Apache Spark (PySpark), Python, SQL, Delta Lake, Databricks SQL.
- Experience with Azure, AWS, or GCP cloud platforms.
- Strong knowledge of data modeling and ETL/ELT development.
- Experience with Git, CI/CD pipelines, and Agile methodologies.
- Excellent leadership, communication, and stakeholder management skills.

Preferred Skills

- Unity Catalog, Structured Streaming, Kafka, Azure Data Factory / AWS Glue, Terraform, MLflow, DataOps practices.
- Snowflake (preferred).

Qualifications

- Bachelor's degree in Computer Science, Information Technology, or a related field.
- Databricks certification is preferred.
- Experience leading enterprise-scale cloud data projects is highly desirable.

#J-18808-Ljbffr

📌 Data Engineer - Lead (City of Sydney)
🏢 eTeam
📍 City of Sydney