15 Oct
|
BULLIT RECRUITMENT SOLUTIONS
|
Sydney
15 Oct
BULLIT RECRUITMENT SOLUTIONS
Sydney
:
Roles and Responsibilities :
- Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
- Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.
- Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
- Implement ETL/ELT processes using Spark (PySpark) and SQL.
- Design and optimize data models to support analytics and reporting workloads.
- Develop and maintain Fabric Data Warehouse solutions using T-SQL.
- Ensure data quality, reliability, and consistency through validation and monitoring.
- Optimize pipeline performance, storage, and capacity utilization in Fabric.
- Implement security, access control (RBAC), and data sensitivity labels.
- Enable and support Power BI semantic models and downstream reporting use cases.
- Monitor, troubleshoot, and resolve data pipeline and performance issues.
- Implement data governance, lineage, and cataloging using Microsoft Purview.
- Support CI/CD and DevOps practices for analytics and data engineering workloads.
- Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.
Skill Experience requirement :
- Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).
- Build and optimise Fabric Lakehouse tables, Warehouses, Pipelines, and Dataflows Gen2.
- Define data modelling standards, governance frameworks, and security patterns.
- Develop reusable ETL/ELT frameworks using PySpark, SQL, and Fabric notebooks.
- Manage CI/CD workflows via Azure DevOps and Git.
- Architect scalable,
cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
- Apply best practices for performance tuning, cost optimisation, and data quality.
- Provide technical leadership and guidance to engineering teams.
- Expertise in Delta Lake, Fabric Data Engineering, ADLS, PySpark/SQL.
- Solid understanding of data modelling and integration with Power BI datasets.
- Strong understanding of metadata management, Purview, access control, and FinOps.
- Certifications: DP-203, Fabric Analytics Engineer (preferred).
· Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
· Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.
· Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
· Implement ETL/ELT processes using Spark (PySpark) and SQL.
· Design and optimize data models to support analytics and reporting workloads.
· Develop and maintain Fabric Data Warehouse solutions using T-SQL.
· Ensure data quality, reliability, and consistency through validation and monitoring.
· Optimize pipeline performance, storage, and capacity utilization in Fabric.
· Implement security, access control (RBAC), and data sensitivity labels.
· Enable and support Power BI semantic models and downstream reporting use cases.
· Monitor, troubleshoot, and resolve data pipeline and performance issues.
· Implement data governance, lineage, and cataloging using Microsoft Purview.
· Support CI/CD and DevOps practices for analytics and data engineering workloads.
· Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.
📌 Microsoft Fabric Data Engineer (Sydney)
🏢 BULLIT RECRUITMENT SOLUTIONS
📍 Sydney