Key Responsibilities

- Design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling.

- Ensure pipelines are resilient, performant, and cost optimised.

- Implement monitoring, alerting, and automated recovery processes.

- Apply data quality checks and reconciliation processes to ensure accuracy and completeness.

- Maintain metadata, lineage, and documentation to support governance requirements.

- Adhere to role-based access control, data privacy, and security standards.

- Contribute to the development and maintenance of ingestion frameworks, transformation layers, and data models.

- Implement and maintain CI/CD pipelines for automated testing and deployment.

- Optimise data storage and retrieval strategies to meet performance needs.

- Work closely with Technical Leads, Data Architects, Analysts, and business stakeholders to deliver fit-for-purpose solutions.

- Participate in code reviews and knowledge-sharing sessions to improve team capability.

- Explore and adopt new tools, frameworks, and techniques to enhance engineering capability.

- Identify opportunities to automate and streamline workflows.





Key Skills & Experience

- Proficient in SQL for data wrangling, transformation, and analysis

- Robust programming skills

- Experience building ETL/ELT pipelines in cloud or distributed environments

- Familiarity with cloud platforms and services

- Knowledge of data modeling concepts

- Experience with data orchestration tools

- Familiar with version control and CI/CD pipelines

- Understanding of data quality, validation, and observability practices

- Works independently with minimal supervision on well-scoped tasks

- Collaborates effectively with stakeholders

- Communicates clearly and documents work for others to maintain or build upon

- Open to code reviews, feedback, and continuous learning

- Able to troubleshoot and resolve data pipeline failures or performance issues

- 2–4 years in a data engineering or closely related role

- Has delivered production-ready pipelines end-to-end

Demonstrated ability to improve code quality, reliability, or performance

📌 Data Engineer ( GCP ) (Sydney)
🏢 Xpt Software Australia
📍 Sydney