*This is an on-site/hybrid role, based out of Sydney. It is not open to overseas applications. *
About the Role:
The Senior Data Engineer is responsible for the design, development, testing, maintenance, and documentation of data pipelines and data platforms in accordance with client requirements, system specifications, and defined technical architecture.
The role undertakes analysis of system requirements and evaluates existing data systems, identifying limitations and deficiencies in processes and methods. The position is accountable for the full lifecycle of data solutions, including requirements gathering, design, implementation, testing, deployment, and ongoing production support.
Key Responsibilities:
Analyse and evaluate client data system requirements
Design data architectures, including lakehouse and medallion patterns
Define data models, schema structures, partitioning strategies, and storage layouts
Develop solutions primarily using Databricks within AWS, Azure, and GCP environments
Design, develop, and maintain data pipeline code using Python, Scala, and SQL
Implement data ingestion, transformation, and delivery processes in accordance with system specifications
Test, debug, and resolve defects in line with established standards and protocols
Data Quality & Governance
Develop and implement data quality frameworks, validation processes, and monitoring mechanisms
Ensure data platforms operate in accordance with defined specifications and governance requirements
Prepare and maintain technical documentation and operational procedures
Develop and manage infrastructure using Infrastructure as Code tools (e.g. Terraform,



CDK, CloudFormation)
Maintain version-controlled infrastructure definitions
Support CI/CD processes for automated deployment of data systems
Design and implement integrations with enterprise systems, including identity, security, and compliance frameworks
System Integration and Improvement
Identify and assess limitations in existing systems and platforms
Design and implement solutions to address integration, performance, and scalability requirements
Technical Strategy & Advisory
Provide technical input into data system design, platform selection, and architecture decisions
Contribute to the development of proposals, including cost estimation and financial evaluation of technology options
Client Enablement and Knowledge Transfer
Work collaboratively with client teams to support implementation and operation of data platforms
Develop and maintain documentation to support ongoing system use and maintenance
Facilitate knowledge transfer to enable client self-sufficiency
Multi-Cloud Capability
The role operates across multi-cloud environments and requires the design of data systems that are portable and scalable. Platforms include:
AWS: Databricks, S3, Glue, Lambda, EventBridge, Redshift, IAM, VPC, KMS
GCP: Databricks,



GCS, BigQuery, Dataflow, Pub/Sub, IAM, VPC
Required Qualifications and Experience
Bachelor's degree in Computer Science, Information Technology, Software Engineering, or a related discipline
OR at least 5 years' relevant professional experience with appropriate vendor certifications
Demonstrated experience delivering and maintaining production data systems
Experience with Databricks in a production setting, including Delta Lake and Unity Catalog
Proficiency in SQL and Python or Scala
Experience in designing and implementing data quality and validation frameworks
Demonstrated ability to communicate technical concepts clearly to non-technical stakeholders
Strong software engineering practices, including testing, version control (Git), and CI/CD
Experience with Infrastructure as Code tools (e.g. Terraform, CloudFormation)
Understanding of distributed systems and data pipeline design
Ability to produce clear and comprehensive technical documentation
Bonus Points
Experience working with Databricks across multiple cloud platforms, along with Genie Spaces experience
Experience with real-time or streaming data systems (e.g. Kafka)
Knowledge of data governance, privacy, and compliance frameworks (e.g. APRA CPG 234)
Exposure to machine learning infrastructure and pipelines
Background in platform engineering or Site Reliability Engineering
Cloud platform certifications (AWS, Azure, or GCP)
Experience contributing to technical knowledge sharing or capability development within teams
#J-*****-Ljbffr

📌 Senior Data Engineer (New South Wales)
🏢 DX1
📍 New South Wales