Job Description

Location : Sydney

n

Mode:Permanent

n

Job Description:

n

Primary or Mandatory skills

n
Scala Spark

n
SQL

n
Python

n

Valuable to have skills

n

Databricks, SQL, Python, Data warehousing concepts

n

Detailed Job description (Role & Responsibility, Skill and Experience range)

n

n
We are looking for a hands on Developer Programmer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL
n
Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices.
n
Proven expertise in designing real time and near real time integrations
n
Operationalize workloads:



jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability.
n

n

Expertise in cloud platforms and related technologies, specifically:

n

Data Platforms (e.g. Databricks) and big data technologies.

n

Real time data processing frameworks Spark Streaming, Apache/Confluent Kafka

n

n
Proficiency in SQL, Python, and data integration tools.
n
Familiarity with Master Data Management solutions
n
Excellent communication and stakeholder management skills.
n
J-18808-Ljbffr

📌 Data Engineer Sydney
🏢 CareCone Group
📍 Sydney