XPT Software Australia Pty Ltd | Contract

Sydney, Australia | Posted on 10/30/2026

- XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company

- XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.

- We have 120+technocrats in Australia working at our clientlocations.

- XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.

- We have served100+ clients globally, fulfilling their onsite-offshoreneeds.

Job Description

Role Overview

We are looking for a Senior Data Engineerwith strong hands-on experience in Azure Databricks and the Microsoft Azuredata ecosystem. The role will design, build, optimise, and support scalabledata pipelines and lakehouse solutions that enable analytics, reporting, datascience, and enterprise decision-making.

What You'll Work On

- Design,build, and optimise scalable batch and streaming data pipelines using AzureDatabricks, PySpark, Python, SQL, and Delta Lake.
- Developreusable ingestion, transformation, validation, and publishing frameworks forlarge-scale enterprise datasets.
- Implementmedallion or layered lakehouse patterns across raw, curated, andconsumption-ready data zones.
- IntegrateAzure Databricks with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and related Azure services.
- Developproduction-grade Databricks notebooks, jobs, workflows, libraries, and clusterconfigurations.
- Createcomplex SQL and PySpark transformations, data-quality checks, reconciliationlogic, and error-handling routines.
- OptimiseSpark workloads, storage layouts, partitioning, file sizes, caching, and queryperformance while controlling platform cost.
- ImplementDelta Lake capabilities such as ACID transactions, schema enforcement, schemaevolution, time travel, and merge operations.
- Buildautomated CI/CD pipelines for code promotion, configuration management,testing, and deployment across environments.




- Applydata security and governance controls using role-based access, secretsmanagement, data masking, auditability, and controlled access patterns.
- Monitordata pipelines, troubleshoot failures, perform root-cause analysis, and deliversustainable production fixes.
- Partnerwith Architects, Business Analysts, Data Analysts, Product Owners, and businessstakeholders to convert requirements into reliable data products.
- Providetechnical leadership, code reviews, engineering standards, and mentoring toother data engineers.

KeySkills and Experience

- 8+years of experience in data engineering, data integration, ETL/ELT, orlarge-scale data platform delivery.
- Stronghands-on experience with Azure Databricks, including notebooks, jobs orworkflows, cluster configuration, debugging, and performance tuning.
- Advancedproficiency in Python, PySpark, and SQL for production-grade data engineering.
- Strongexperience with Delta Lake and lakehouse architecture, including reliableincremental processing and data-quality controls.
- Practicalexperience with Azure Data Lake Storage Gen2 and Azure Data Factory.
- Experienceimplementing CI/CD using Azure DevOps, Git, branching strategies, automatedtesting, and setting-based deployment controls.
- Strongunderstanding of dimensional modelling, data warehousing, data lake design,metadata, lineage, partitioning, and file formats such as Parquet.
- Experiencewith orchestration, monitoring, alerting, operational support, incidentanalysis, and service-level expectations.
- Understandingof Azure identity, Key Vault, managed identities, networking, privateendpoints, and secure data-platform patterns.
- Strongcommunication, documentation, problem-solving, code-review, and stakeholdercollaboration skills.

Good to Have

- Experiencewith Microsoft Purview, Unity Catalog, Azure Synapse Analytics, Event Hubs, orStructured Streaming.
- Knowledgeof infrastructure as code using Terraform or Bicep.
- Experiencewith dbt, Airflow, Kafka, Power BI, or machine-learning data pipelines.
- Experiencein telecommunications or another high-volume enterprise environment.

📌 Data Engineer (Sydney)
🏢 Xpt Software Australia
📍 Sydney