Role: MS Fabric Data Engineer

Location: Sydney

Job Type: Permanent

Experience: 5+ yrs

:

Roles and Responsibilities:

- Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
- Develop and manage Lakehouse and One Lake architectures for structured and unstructured data.
- Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
- Implement ETL/ELT processes using Spark (Pyspark) and SQL.
- Design and optimize data models to support analytics and reporting workloads.
- Develop and maintain Fabric Data Warehouse solutions using T-SQL.
- Ensure data quality, reliability, and consistency through validation and monitoring.
- Optimize pipeline performance, storage, and capacity utilization in Fabric.
- Implement security, access control (RBAC), and data sensitivity labels.
- Enable and support Power BI semantic models and downstream reporting use cases.
- Monitor, troubleshoot, and resolve data pipeline and performance issues.
- Implement data governance, lineage, and cataloguing using Microsoft Purview.
- Support CI/CD and DevOps practices for analytics and data engineering workloads.
- Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement:

- Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).
- Build and optimise Fabric Lakehouse tables, Warehouses, Pipelines, and Dataflows Gen2.
- Define data modelling standards, governance frameworks, and security patterns.
- Develop reusable ETL/ELT frameworks using PySpark, SQL,



and Fabric notebooks.
- Manage CI/CD workflows via Azure DevOps and Git.
- Architect scalable, cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
- Apply best practices for performance tuning, cost optimisation, and data quality.
- Provide technical leadership and guidance to engineering teams.
- Expertise in Delta Lake, Fabric Data Engineering, ADLS, PySpark/SQL.
- Strong understanding of data modelling and integration with Power BI datasets.
- Strong understanding of metadata management, Purview, access control, and FinOps.
- Certifications: DP-203, Fabric Analytics Engineer (preferred).

· Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.

· Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.

· Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.

· Implement ETL/ELT processes using Spark (PySpark) and SQL.

· Design and optimize data models to support analytics and reporting workloads.

· Develop and maintain Fabric Data Warehouse solutions using T-SQL.

· Ensure data quality, reliability, and consistency through validation and monitoring.

· Optimize pipeline performance, storage,



and capacity utilization in Fabric.

· Implement security, access control (RBAC), and data sensitivity labels.

· Enable and support Power BI semantic models and downstream reporting use cases.

· Monitor, troubleshoot, and resolve data pipeline and performance issues.

· Implement data governance, lineage, and cataloging using Microsoft Purview.

· Support CI/CD and DevOps practices for analytics and data engineering workloads.

· Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement

· Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).

· Build and optimise Fabric Lakehouse tables, Warehouses, Pipelines, and Dataflows Gen2.

· Define data modelling standards, governance frameworks, and security patterns.

· Develop reusable ETL/ELT frameworks using PySpark, SQL, and Fabric notebooks.

· Manage CI/CD workflows via Azure DevOps and Git.

· Architect scalable, cost-efficient data solutions integrating Fabric with Azure/enterprise systems.

· Apply best practices for performance tuning, cost optimisation, and data quality.

· Provide technical leadership and guidance to engineering teams.

· Expertise in Delta Lake, Fabric Data Engineering, ADLS, PySpark/SQL.

· Robust understanding of data modelling and integration with Power BI datasets.

· Strong understanding of metadata management, Purview, access control, and FinOps.

· Certifications: DP-203, Fabric Analytics Engineer (preferred).

📌 Data Engineer MS Fabrics (Sydney)
🏢 The HR Ally
📍 Sydney