- Ability to proactively collect client requirements and design solution
- Minimum of 12-15 years overall experience with previous experience as Data Engineer, Solution designer
Location:
Sydney or Melbourne with Full working rights
Responsibilities:
- Architecture & Platform Strategy:
Fabric Ecosystem Design: Design and deploy the end-to-end Microsoft Fabric tenant topology, capacity sizing (F-SKU), workspace structures, and OneLake organization.
- Medallion Implementation: Architect and govern the Bronze (raw landing), Silver (cleansed & conformed Delta tables), and Gold (analytical semantic models) data tiers.
- Data Integration Strategy: Establish ingestion patterns for real-time streaming, batch processing, Change Data Capture (CDC), and API-driven integrations.
- Data Quality & Lineage: Define automated data verification, deduplication, and lineage tracking standards across all pipeline layers.
Required Skills
- Data Storage Formats:
Deep knowledge of Delta Lake storage format, parquet optimization, partitioning strategies, and ACID transaction rules.
- Transformations & Querying:
Mastery of PySpark, Spark SQL, T-SQL, and Data Factory orchestration.