Job Description

Overview

n

Job Description for Lead Data Engineer in Sydney

n

Responsibilities

n

n
- Leads data engineering initiatives with a focus on data warehousing, data analysis, visualization and data modeling.
n
- Designs and implements data solutions and pipelines using Python, Scala and SQL.
n
- Works with cloud platforms (Azure and Databricks) to build and maintain data infrastructure.
n
- Leverages Big Data technologies (Cloudera: HDFS, Spark, Hive, Kafka) and Apache Airflow for data workflows.
n
- Applies DevOps practices with Azure DevOps (Git, Repos, Pipelines), Git, Terraform and Docker.
n
- Supports reporting needs using SQL Server Reporting Services and Power BI.
n

n

Qualifications

n

n
- 10+ years of experience in software engineering,



cloud computing, Databricks and Big Data.
n
- Data Engineering, Data Warehousing, Solution Design, Data Analysis, Visualization and Data Modeling.
n
- Programming: Python, Scala, SQL.
n
- Cloud: Azure and Databricks.
n
- Big Data: Cloudera (HDFS, Spark, Hive, Kafka), Apache Airflow.
n
- DevOps: Azure DevOps (Git, Repos, Pipelines), Git, Terraform, Docker.
n
- Reporting: SQL Server Reporting Services, Power BI.
n
- Certified DevOps Engineer Expert, Azure Cloud, Azure Data Lake.
n

#J-18808-Ljbffr

📌 Lead Data Engineer (New South Wales)
🏢 Pyramid Global Technologies
📍 New South Wales