Overview

Job Description for Lead Data Engineer in Sydney

Responsibilities
Leads data engineering initiatives with a focus on data warehousing, data analysis, visualization and data modeling.
Designs and implements data solutions and pipelines using Python, Scala and SQL.
Works with cloud platforms (Azure and Databricks) to build and maintain data infrastructure.
Leverages Big Data technologies (Cloudera: HDFS, Spark, Hive, Kafka) and Apache Airflow for data workflows.
Applies DevOps practices with Azure DevOps (Git, Repos, Pipelines), Git, Terraform and Docker.
Supports reporting needs using SQL Server Reporting Services and Power BI.

Qualifications
10+ years of experience in software engineering, cloud computing, Databricks and Big Data.
Data Engineering, Data Warehousing, Solution Design, Data Analysis, Visualization and Data Modeling.
Programming: Python, Scala, SQL.
Cloud: Azure and Databricks.
Big Data: Cloudera (HDFS, Spark, Hive, Kafka), Apache Airflow.
DevOps: Azure DevOps (Git, Repos, Pipelines), Git, Terraform, Docker.
Reporting: SQL Server Reporting Services, Power BI.
Certified DevOps Engineer Expert, Azure Cloud, Azure Data Lake.
J-18808-Ljbffr

📌 Lead Data Engineer City Of Sydney
🏢 Pyramid Global Technologies
📍 City of Sydney