We are looking for an experienced Senior Azure Data Engineer / Lead with 8+ years of software/data engineering experience and strong expertise in Azure, Databricks, Python, SQL and Apache Spark.
The ideal candidate will have hands-on experience designing and developing complex data pipelines, distributed data processing solutions, ETL frameworks and large-scale data platforms within Agile environments.
Key Responsibilities & Requirements
- 8+ years of experience in software/data engineering and development, preferably within Agile environments.
- Strong hands-on expertise in Python and SQL; Java experience is an advantage.
- Design, develop and optimize complex data pipelines and ETL workflows.
- Strong experience with Apache PySpark, Databricks, Azure Synapse, Azure Data Factory (ADF) and Azure Functions.
- Hands-on experience with data modelling, data warehousing and large-scale ETL architectures.
- Experience working with MS SQL, ADLS Gen2 and NoSQL/data storage technologies.
- Good understanding of object storage and Big Data architectures, particularly Azure Data Lake Storage Gen2.
- Experience managing and optimizing distributed systems and clusters for large-scale/batch data processing.
- Knowledge of MLOps architecture and practices.
- Strong understanding of modern data modelling concepts and enterprise data warehouse architecture.
- Familiarity with Azure DevOps, Git and modern CI/CD practices.
- Good knowledge of ETL and data warehousing methodologies.
- Informatica experience is an advantage.
- Ability to work closely with architects, developers, data teams and business stakeholders in an Agile workplace.
📌 Senior Azure Data Engineer – Databricks -Sydney Only
🏢 The HR Ally
📍 Sydney
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.