XPT Software Australia Pty Ltd | Contract

Sydney, Australia | Posted on 10/30/2026

XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company

XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.

We have 120+technocrats in Australia working at our clientlocations.

XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.

We have served100+ clients globally, fulfilling their onsite-offshoreneeds.

Job Description Role Overview We are looking for a Senior Data Engineerwith strong hands-on experience in Azure Databricks and the Microsoft Azuredata ecosystem. The role will design, build, optimise, and support scalabledata pipelines and lakehouse solutions that enable analytics, reporting, datascience, and enterprise decision-making.

What You'll Work On

Design,build, and optimise scalable batch and streaming data pipelines using AzureDatabricks, PySpark, Python, SQL, and Delta Lake.

Developreusable ingestion, transformation, validation, and publishing frameworks forlarge-scale enterprise datasets.

Implementmedallion or layered lakehouse patterns across raw, curated, andconsumption-ready data zones.

IntegrateAzure Databricks with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and related Azure services.

Developproduction-grade Databricks notebooks, jobs, workflows, libraries, and clusterconfigurations.

Createcomplex SQL and PySpark transformations, data-quality checks, reconciliationlogic, and error-handling routines.

OptimiseSpark workloads, storage layouts, partitioning, file sizes, caching, and queryperformance while controlling platform cost.

ImplementDelta Lake capabilities such as ACID transactions, schema enforcement, schemaevolution, time travel, and merge operations.

Buildautomated CI/CD pipelines for code promotion, configuration management,testing, and deployment across environments.





Applydata security and governance controls using role-based access, secretsmanagement, data masking, auditability, and controlled access patterns.

Monitordata pipelines, troubleshoot failures, perform root-cause analysis, and deliversustainable production fixes.

Partnerwith Architects, Business Analysts, Data Analysts, Product Owners, and businessstakeholders to convert requirements into reliable data products.

Providetechnical leadership, code reviews, engineering standards, and mentoring toother data engineers.

KeySkills and Experience

8+years of experience in data engineering, data integration, ETL/ELT, orlarge-scale data platform delivery.

Stronghands-on experience with Azure Databricks, including notebooks, jobs orworkflows, cluster configuration, debugging, and performance tuning.

Advancedproficiency in Python, PySpark, and SQL for production-grade data engineering.

Strongexperience with Delta Lake and lakehouse architecture, including reliableincremental processing and data-quality controls.

Practicalexperience with Azure Data Lake Storage Gen2 and Azure Data Factory.

Experienceimplementing CI/CD using Azure DevOps, Git, branching strategies, automatedtesting, and environment-based deployment controls.

Strongunderstanding of dimensional modelling, data warehousing, data lake design,metadata, lineage, partitioning, and file formats such as Parquet.

Experiencewith orchestration, monitoring, alerting, operational support, incidentanalysis, and service-level expectations.

Understandingof Azure identity, Key Vault, managed identities, networking, privateendpoints, and secure data-platform patterns.

Strongcommunication, documentation, problem-solving, code-review, and stakeholdercollaboration skills.

Valuable to Have

Experiencewith Microsoft Purview, Unity Catalog, Azure Synapse Analytics, Event Hubs, orStructured Streaming.

Knowledgeof infrastructure as code using Terraform or Bicep.

Experiencewith dbt, Airflow, Kafka, Power BI, or machine-learning data pipelines.

Experiencein telecommunications or another high-volume enterprise environment.

#J-18808-Ljbffr

📌 Data Engineer (Haymarket)
🏢 Xpt Software Australia
📍 Haymarket