Role Overview

We are looking for an experienced Data Engineer with strong expertise in Databricks, Azure Data Factory (ADF), Python, SQL, and Apache Spark. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data processing solutions.

Key Responsibilities
Design and develop scalable data pipelines using Databricks and Apache Spark.
Build and maintain ETL/ELT workflows using Azure Data Factory (ADF).
Develop efficient data processing solutions using Python and PySpark.
Write complex SQL queries for data transformation, validation, and analysis.
Work with large datasets and optimize Spark jobs and data pipelines for performance.
Implement data quality, validation, monitoring, and error-handling processes.
Collaborate with Data Architects, Analysts, Developers, and business stakeholders.
Troubleshoot data pipeline failures and resolve performance issues.
Follow best practices for data security,



governance, and documentation.

Required Skills
Solid hands-on experience with Databricks.
Proficiency in Apache Spark / PySpark.
Robust programming skills in Python.
Advanced knowledge of SQL.
Hands-on experience with Azure Data Factory (ADF).
Positive understanding of ETL/ELT concepts and data warehousing.
Experience working with cloud-based data platforms, preferably Microsoft Azure.
Strong problem-solving and communication skills.

Good to Have
Experience with Delta Lake / Delta Tables.
Knowledge of Azure Data Lake Storage (ADLS).
Experience with CI/CD and version control tools such as Git/Azure DevOps.
Understanding of data governance and security practices.

Pay: $800.00 – $900.00 per day

Work Location: Hybrid remote in Melbourne VIC

📌 Data Engineer – Databricks Melbourne (Australia)
🏢 Diksha Technologies
📍 Australia