Data Engineer – Databricks / PySpark
Canberra | Hybrid | Baseline Clearance Required
$105–$120 per hour incl. super

We are seeking an experienced Data Engineer to support the design, migration and operation of data solutions within a Federal Government workplace.
This is a hands-on engineering role focused on Databricks, PySpark, Spark SQL, Delta Lake and SAS-to-Databricks migration.
You will work within the Schools Data Production Team to build scalable data pipelines, migrate existing SAS processes and establish secure, maintainable and well-documented data engineering practices.

Key responsibilities
Design, build and maintain scalable Databricks data pipelines
Migrate SAS code, workflows and scheduled processes to Databricks
Develop solutions using PySpark, Spark SQL and advanced SQL
Build ETL/ELT pipelines and medallion-layer data models
Implement Delta Lake solutions
Establish source control, deployment and testing practices
Work with Azure DevOps, Git and CI/CD
Implement automated testing, monitoring and performance optimisation
Apply data security, privacy, metadata and governance controls




Produce transparent technical documentation and support knowledge transfer

Essential experience
Solid Databricks, PySpark, Spark SQL and SQL capability
Demonstrated experience designing scalable Databricks solutions
Experience building maintainable ETL/ELT pipelines
Experience with Delta Lake and medallion architecture
Strong experience with Azure DevOps, Git and CI/CD
Experience with automated testing, monitoring and performance optimisation
Ability to produce secure, maintainable and well-documented data solutions

Highly desirable / strongly preferred:
Demonstrated experience migrating SAS code, workflows and reporting processes to Databricks

EligibilityApplicants must:
Be able to work from the Canberra office under hybrid arrangements
Hold a current Baseline security clearance or higher

Apply
Respond with your tailored CV. Suitable candidates will be asked to complete and application pack, including a One-page pitch for the role.
Candidates not meeting requirements abve, Citizenship and Security clearance requirements, will not be contacted.

📌 Data Engineer Canberra
🏢 Canberra Consulting
📍 Canberra