- 5+ years of Data Engineering experience in enterprise environments.
- 3+ years of hands‑on experience working with Azure Databricks and Apache Spark.
- Proven experience delivering data solutions on Azure cloud platforms.
- Experience designing and implementing scalable data pipelines and data lake architectures.
- Strong experience with Agile delivery methodologies.
- Ability to work independently and contribute immediately within a project‑based environment.
- Robust stakeholder management and communication skills.
Key Responsibilities
- Design, develop, and optimize data pipelines using Azure Databricks and Apache Spark.
- Build and maintain robust ETL/ELT frameworks to ingest, transform, and load data from multiple source systems.
- Develop scalable data solutions leveraging Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics.
- Implement data quality,
validation, and monitoring frameworks to ensure data integrity and reliability.
- Collaborate with solution architects, business analysts, data scientists, and stakeholders to deliver project outcomes.
- Optimize Spark workloads, cluster configurations, and data processing performance.
- Support deployment activities through CI/CD pipelines and DevOps practices.
- Troubleshoot production issues and provide timely resolution of data platform incidents.
- Contribute to technical documentation, knowledge transfer, and operational handover activities.
- Ensure adherence to security, governance, compliance, and enterprise data standards.
Duration: 6 Months and possible extension
Eligibility: Australian/NZ Citizens/PR Holders only