Job Title :Fabric Data Engineer

Location :Sydeny

Mode:Contract

1. Key Responsibilities

Microsoft Fabric Development
- Design and implement end-to-end data engineering solutions using Microsoft Fabric.
- Develop and maintain Fabric Lakehouse and Warehouse solutions.
- Work with OneLake for centralized data storage and management.
- Implement data ingestion and transformation processes using Fabric Data Factory.
- Develop and manage Data Pipelines, Dataflows Gen2, and Notebooks.
- Implement Medallion Architecture (Bronze, Silver, and Gold layers).
- Optimize data storage, processing, and query performance.

ETL / ELT Development
- Design, develop, and maintain scalable ETL/ELT pipelines.
- Extract data from multiple sources, including databases, APIs, files, and cloud platforms.
- Perform data cleansing, transformation, validation, and integration.
- Implement incremental data loading and change data capture (CDC) where required.
- Monitor pipeline execution and troubleshoot data processing issues.
- Ensure data quality, consistency, and reliability.

SQL and Python Development
- Write complex SQL queries, stored procedures, and views.
- Develop data transformation logic using Python and PySpark.
- Work with Spark SQL for large-scale data processing.
- Optimize SQL queries and Spark jobs for better performance.
- Handle structured, semi-structured, and unstructured data.

Data Architecture and Modeling
- Design scalable data warehouse and Lakehouse architectures.
- Develop fact tables, dimension tables, and star schemas.




- Implement Slowly Changing Dimensions (SCD Type 1 and Type 2).
- Build reusable data models for reporting and analytics.
- Ensure data governance, security, and compliance.

Power BI and Reporting Integration
- Integrate Microsoft Fabric data solutions with Power BI.
- Prepare curated datasets for business intelligence and reporting.
- Develop semantic models and support Direct Lake connectivity.
- Collaborate with Power BI developers and business analysts.
- Ensure optimized data availability for reporting requirements.

Performance Optimization and Monitoring
- Monitor and troubleshoot data pipelines and notebook executions.
- Optimize Spark workloads and SQL queries.
- Manage pipeline scheduling and dependencies.
- Implement error handling, logging, and monitoring.
- Ensure high availability and reliability of data solutions.

1. Required Technical Skills

Technical Skill

Required Knowledge

Microsoft Fabric

Lakehouse, Warehouse, OneLake

Fabric Data Factory

Pipelines, Dataflows Gen2

PySpark

Data processing and transformations

Python

Data engineering and automation

SQL

Joins, CTEs, stored procedures, optimization

Apache Spark

Spark SQL, distributed processing

Delta Lake

Delta tables, ACID transactions

Data Architecture

Medallion Architecture

Data Modeling

Star schema, fact and dimension tables

Power BI

Semantic models, Direct Lake

ETL/ELT

Data ingestion and transformation

Azure

Azure Data Lake Storage, Azure Data Factory

📌 Fabric Data Engineer (Sydney)
🏢 CareCone Group
📍 Sydney