Location : Sydney
Mode:Permanent
Job Description:
Primary or Mandatory skills
2. Scala Spark
3. SQL
4. Python
Positive to have skills
Databricks, SQL, Python, Data warehousing concepts
Detailed Job description (Role & Responsibility, Skill and Experience range)
We are looking for a hands on Developer Programmer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL
Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices.
Proven expertise in designing real time and near real time integrations
Operationalize workloads: jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability.
Expertise in cloud platforms and related technologies, specifically:
Data Platforms (e.g. Databricks) and big data technologies.
Real time data processing frameworks Spark Streaming, Apache/Confluent Kafka
Proficiency in SQL, Python, and data integration tools.
Familiarity with Master Data Management solutions
Excellent communication and stakeholder management skills.
#J-*****-Ljbffr

📌 Data Engineer (New South Wales)
🏢 CareCone Group
📍 New South Wales