Job Description

Mandatory Skill Set: SQL + Python + PySpark + Azure Synapse

n

Role Overview

n

We are seeking a highly experienced Lead Data Engineer to drive the strategy, architecture, delivery, and optimization of enterprise-scale data platforms on Microsoft Azure. The ideal candidate will combine deep technical expertise with strong leadership, stakeholder management, and delivery governance capabilities to build scalable, secure, and cost-efficient data solutions.

n

Must-Have Skills

n

n
- Strong expertise in SQL, Python, and PySpark
n
- Extensive experience with Azure Data Factory (ADF) and Azure Synapse Analytics.
n
- Hands-on experience with Delta Lake architecture and contemporary data lakehouse implementations.
n
- Advanced knowledge of Data Modeling and Data Warehousing concepts.
n
- Strong exposure to the Microsoft Fabric ecosystem.
n
- Proven expertise in designing cloud-native data architectures and enterprise-scale solutions.
n
- Strong design thinking and problem-solving capabilities.
n
- Working knowledge of CI/CD pipelines and DevOps practices.
n
- Experience in cloud cost optimization, performance tuning, and resource governance.
n

n

Key Responsibilities

n

n
- Lead and own the end-to-end delivery of enterprise data platforms, including architecture, development, deployment, and optimization.
n
- Define and drive the data platform roadmap, ensuring alignment with business objectives and strategic priorities.
n
- Architect and govern scalable data pipelines, data models, data warehouses, and lakehouse solutions using Azure services.
n
- Provide technical leadership to Data Engineering teams through design reviews,



code reviews, and implementation of engineering best practices.
n
- Drive the adoption and implementation of Microsoft Fabric, APIs, Azure Functions, and Python-based solutions.
n
- Ensure platform scalability, reliability, security, and operational excellence while optimizing cloud costs.
n
- Establish and enforce data governance, data quality, security, and compliance standards.
n
- Implement and drive CI/CD, DevOps, and Agile delivery practices across the data engineering landscape.
n
- Collaborate with business stakeholders, architects, and technology teams to translate business requirements into scalable technical solutions.
n
- Mentor, coach, and develop Data Engineering talent, fostering a culture of continuous learning and innovation.
n
- Leverage GitHub Copilot and AI-assisted engineering tools to improve productivity and solution quality.
n
- Drive innovation by evaluating emerging technologies and identifying opportunities for platform modernization.
n

n

Experience

n

n
- 10+ years of overall IT experience, including 10+ years in Data Engineering.
n
- Minimum 5+ years of leadership experience managing large-scale Data Engineering programs and teams.
n




- Proven track record of leading cross-functional teams and delivering complex enterprise data transformation initiatives.
n
- Experience building, hiring, and scaling high-performing Data Engineering teams.
n
- Experience working within multi-vendor and globally distributed delivery environments.
n
- Strong experience in stakeholder management, delivery governance, and executive-level communication.
n

n

Preferred Skills

n

n
- Experience building and managing enterprise APIs and Azure Function Apps.
n
- Exposure to framework-based data engineering development approaches.
n
- Experience in solution architecture, stakeholder engagement, and delivery governance.
n
- Azure certifications such as:
n
- Azure Data Engineer Associate
n
- Azure Solutions Architect Expert
n

n

Familiarity with MLOps, Data Science, and AI-driven analytics solutions

n

Hands-on experience with Power BI and enterprise reporting platforms

n

Experience leveraging AI and Generative AI capabilities within Data Engineering ecosystems

n

Key Leadership Traits

n

n
- Strong ownership mindset with a focus on accountability and business outcomes.
n
- Excellent communication, stakeholder management, and influencing skills.
n
- Ability to balance technical leadership with delivery and people management responsibilities.
n
- Strategic thinker with a strong focus on execution excellence.
n
- Strong conflict resolution and cross-functional collaboration skills.
n
- Adaptability and resilience in fast-paced, evolving environments.
n
- Passion for mentoring teams and fostering a culture of innovation and continuous improvement.
n

📌 Lead Data Engineer (Victoria)
🏢 Xpt Software Australia
📍 Victoria