Data Engineer – Databricks / PySpark
Canberra | Hybrid | Baseline Clearance Required
$105–$120 per hour incl. super

We are seeking an experienced Data Engineer to support the design, migration and operation of data solutions within a Federal Government environment.
This is a hands-on engineering role focused on Databricks, PySpark, Spark SQL, Delta Lake and SAS-to-Databricks migration.
You will work within the Schools Data Production Team to build scalable data pipelines, migrate existing SAS processes and establish secure, maintainable and well-documented data engineering practices.

Key responsibilities

- Design, build and maintain scalable Databricks data pipelines
- Migrate SAS code, workflows and scheduled processes to Databricks
- Develop solutions using PySpark, Spark SQL and advanced SQL
- Build ETL/ELT pipelines and medallion-layer data models
- Implement Delta Lake solutions
- Establish source control, deployment and testing practices
- Work with Azure DevOps, Git and CI/CD
- Implement automated testing, monitoring and performance optimisation
- Apply data security, privacy, metadata and governance controls




- Produce transparent technical documentation and support knowledge transfer

Essential experience

- Strong Databricks, PySpark, Spark SQL and SQL capability
- Demonstrated experience designing scalable Databricks solutions
- Experience building maintainable ETL/ELT pipelines
- Experience with Delta Lake and medallion architecture
- Strong experience with Azure DevOps, Git and CI/CD
- Experience with automated testing, monitoring and performance optimisation
- Ability to produce secure, maintainable and well-documented data solutions

Highly desirable / strongly preferred:

- Demonstrated experience migrating SAS code, workflows and reporting processes to Databricks

EligibilityApplicants must:

- Be able to work from the Canberra office under hybrid arrangements
- Hold a current Baseline security clearance or higher

Apply
Respond with your tailored CV. Suitable candidates will be asked to complete and application pack, including a One-page pitch for the role.
Candidates not meeting requirements abve, Citizenship and Security clearance requirements, will not be contacted.

📌 Data Engineer (Canberra)
🏢 Canberra Consulting
📍 Canberra