Job Description
We are seeking an experienced Senior Data Engineer for an initial three-month contract, with strong long-term potential and the opportunity to transition into a permanent position for the right person.
n
This position will suit a highly capable Data Engineer who combines strong Microsoft Azure and Fabric experience with an understanding of how reliable, governed data is prepared and delivered to analytics, artificial intelligence and agent-based solutions through Microsoft Foundry.
n
The prospect:
n
You will work closely with architects, developers, analysts and business stakeholders to build scalable data pipelines, lakehouse and warehouse solutions, and trusted enterprise data products. A major focus will be integrating structured and unstructured organisational data with Microsoft Foundry to support generative AI, retrieval-augmented generation and intelligent agent solutions.
n
The initial contract will provide both parties with the opportunity to establish a successful working relationship, with a genuine pathway into an ongoing permanent position.
n
Key responsibilities:
n
n
Design, develop and maintain scalable data pipelines across Microsoft Azure and Microsoft Fabric.
n
Build modern lakehouse, warehouse and data integration solutions using Fabric, OneLake, Azure Data Factory, Azure Data Lake and related Microsoft technologies.
n
Develop robust data transformation processes using SQL, Python and PySpark.
n
Prepare and optimise structured and unstructured data for analytics, generative AI and Microsoft Foundry solutions.
n
Integrate enterprise data with Microsoft Foundry models, agents, knowledge sources and APIs.
n
Support retrieval-augmented generation solutions using Azure AI Search, vector indexing, embeddings and enterprise knowledge repositories.
n
Implement data quality, lineage, security, monitoring and governance controls.
n
Work with Microsoft Purview,



Entra ID and role-based access controls to protect sensitive information.
n
Establish automated deployment and testing processes using Azure DevOps, Git and CI/CD.
n
Troubleshoot performance, reliability and data-quality issues across complex production environments.
n
Translate business requirements into practical, scalable and supportable technical solutions.
n
n
Required experience:
n
n
Extensive commercial experience as a Data Engineer within Microsoft-based enterprise environments.
n
Strong hands‐on experience with Microsoft Fabric, Azure Data Factory, OneLake, Azure Data Lake, Synapse Analytics or Azure Databricks.
n
Advanced SQL skills and commercial experience with Python or PySpark.
n
Experience designing data pipelines, lakehouse architectures, data warehouses and dimensional models.
n
Practical experience with Microsoft Foundry, formerly Azure AI Foundry, or closely related Azure AI services.
n
Understanding of generative AI, AI agents, RAG, vector databases, embeddings and enterprise search.
n
Experience integrating data platforms with APIs, applications and cloud-based services.
n
Strong knowledge of data security, governance, privacy and production support practices.
n
Excellent communication skills and the ability to work effectively with technical teams and business stakeholders.
n
n
Highly regarded:
n
n
Experience with Microsoft Foundry Agent Service and Foundry IQ.
n
Microsoft Fabric, Azure Data Engineering or Azure AI certifications.
n
Experience with Power BI, Microsoft 365 Copilot, Power Platform or Microsoft Purview.
n
Knowledge of infrastructure as code using Bicep or Terraform.
n
Experience delivering AI-enabled data solutions within a large, regulated or complex organisation.
n
n
Additional information
n
n
Immediate start
n
Short-term contract with high potential for long-term opportunity
n
Exciting opportunity for a seasoned developer
n
#J-*****-Ljbffr

📌 Senior Data Engineer - Azure (Victoria)
🏢 Milestone It
📍 Victoria