06 Aug
|
VOCUS
|
Melbourne
We are seeking a skilled and proactive Data Engineer to join Vocus Data Platform team and contribute to the development, execution and maintenance of our end-to-end data pipelines.
In this role, you will collaborate with stakeholders and senior team members to implement scalable architectures across AWS and Databricks. You will focus on building robust ETL/ELT process, optimising existing pipelines and ensuring reliable data delivery across the organisation.
Key Responsibilities Designing, building and supporting enterprise-scale data platforms and data integration solutions in Databricks
Advanced SQL development skills and strong proficiency in Python, PySpark, and modern data engineering frameworks
Designing and implementing reusable, metadata-driven ingestion frameworks, as well as establishing engineering standards, reusable patterns, and platform accelerators
Working with cloud-based data platforms including AWS and Databricks, with hands-on experience in technologies such as Redshift, Databricks, Delta Lake, S3, Glue, Lambda, and MWAA
Dimensional modelling, data warehousing concepts,
and supporting analytical data structures (such as Medallion architecture)
Designing and implementing data quality frameworks, automated validation controls, and data observability practice
Develop and maintain reliable batch, near real-time, and real-time ingestion pipelines leveraging metadata-driven frameworks in Databricks
Manage workflow orchestration using AWS Airflow (MWAA), implement efficient serverless AWS Lambda and Job pipelines in Databricks.
Skills &
Experience 5+ years of hands-on Data Engineering (Databricks Preferred)
Solid core experience with AWS Services
Solid hands-on experience in building job pipelines, streaming, and managing cluster configurations within the Databricks ecosystem
Strong production-grade Python and PySpark skills
Advanced SQL for complex querying, data manipulation, and expertise in modern data warehouse/lakehouse data modelling techniques
Good knowledge and practical experience with CI/CD pipelines and infrastructure-as-code using Terraform.
📌 Data Platform Engineer (Melbourne)
🏢 VOCUS
📍 Melbourne