Build scalable data pipelines using Azure Databricks, Spark, Scala, SQL and Python within a large enterprise data environment.

16th October, 2026

Company Description

Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, infrastructure, cybersecurity, business applications, and digital transformation programs.

We partner with government and enterprise organisations to build high-performing technology teams that deliver scalable, efficient, and future-ready business solutions.

The Role

We are seeking an experienced Data Engineer to join a large-scale enterprise data environment in Sydney.

This is a hands-on engineering role focused on designing, building, and optimising high-throughput data pipelines using Azure Databricks, Spark (Scala), SQL and Python.

You will be responsible for delivering scalable ETL/ELT pipelines, supporting real-time and near-real-time integrations, and developing modern data solutions using Medallion Architecture and Delta Lake best practices.

Working closely with data, engineering and business stakeholders, you will also help operationalise workloads through CI/CD, scheduling, secrets management and observability.

Key Responsibilities

- Design, build and optimise high-throughput data pipelines using Azure Databricks.
- Develop scalable ETL/ELT pipelines using Spark (Scala), SQL and Python.
- Implement data solutions following Medallion Architecture across Bronze, Silver and Gold layers.
- Apply Delta Lake principles and best practices across data workloads.




- Design and develop real-time and near-real-time data integrations.
- Develop data processing solutions using Spark Streaming and Apache/Confluent Kafka.
- Build and manage Databricks jobs, workflows and scheduling processes.
- Support CI/CD pipelines using Git and tools such as Azure DevOps.
- Implement appropriate secrets management and observability practices.
- Work across cloud platforms including Azure and AWS.
- Support large-scale data processing using technologies including Spark, Hive, Impala and Sqoop.
- Collaborate with technical, business and delivery stakeholders to deliver effective data solutions.

16th October, 2026

Company Description

Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, infrastructure, cybersecurity, business applications, and digital transformation programs.

We partner with government and enterprise organisations to build high-performing technology teams that deliver scalable, efficient, and future-ready business solutions.

The Role

We are seeking an experienced Data Engineer to join a large-scale enterprise data environment in Sydney.

This is a hands-on engineering role focused on designing, building,



and optimising high-throughput data pipelines using Azure Databricks, Spark (Scala), SQL and Python.

You will be responsible for delivering scalable ETL/ELT pipelines, supporting real-time and near-real-time integrations, and developing modern data solutions using Medallion Architecture and Delta Lake best practices.

Working closely with data, engineering and business stakeholders, you will also help operationalise workloads through CI/CD, scheduling, secrets management and observability.

Key Responsibilities

- Design, build and optimise high-throughput data pipelines using Azure Databricks.
- Develop scalable ETL/ELT pipelines using Spark (Scala), SQL and Python.
- Implement data solutions following Medallion Architecture across Bronze, Silver and Gold layers.
- Apply Delta Lake principles and best practices across data workloads.
- Design and develop real-time and near-real-time data integrations.
- Develop data processing solutions using Spark Streaming and Apache/Confluent Kafka.
- Build and manage Databricks jobs, workflows and scheduling processes.
- Support CI/CD pipelines using Git and tools such as Azure DevOps.
- Implement appropriate secrets management and observability practices.
- Work across cloud platforms including Azure and AWS.
- Support large-scale data processing using technologies including Spark, Hive, Impala and Sqoop.
- Collaborate with technical, business and delivery stakeholders to deliver effective data solutions.

What You’ll Need

- Robust hands-on experience with Azure Databricks.
- Strong commercial experience with Spark using Scala.
- Strong SQL skills.
- Strong Python skills.
- Exp

📌 Sr. Data Engineer (Sydney)
🏢 Showtime Consulting
📍 Sydney