- What Senior Platform Reliability Engineer AI D...

Showing 1650 Senior Platform Reliability Engineer AI Data Fabric jobs in Sydney

Sorry to hear that! What didn’t you like about this job?

Wrong location

Too few details / Lack of information

Job appears outdated

Salary information missing or unclear

Other (please specify).

Please specify...

Verification *

Thank you for your feedback

Your input helps us improve our service.

- Opportunities to make discounted purchases on a great range of products
- Access to Udemy an online training platform with over 5,500 courses
- Based at Homebush West, free in office parking or a 10 minute walk from Flemington station

As Australia’s largest retailer of lifestyle products, with nearly 300 stores worldwide, Harvey Norman® has been a part of almost every Australian’s shopping experience. Whether getting the latest electronic gadgets, or furnishing your first home, Harvey Norman® is the place where you can shop with confidence.

About the role

In this role, you will design, build and maintain reliable data pipelines, datasets and integration solutions that support AI, intelligent automation and Modern Workplace initiatives. Working closely with product, engineering and business teams, you will create secure, governed and reusable data foundations that enable scalable technology outcomes.

You will;

- Design, build and maintain reliable data pipelines, datasets and integration solutions.
- Manage data ingestion, transformation, validation, enrichment and controlled access.
- Develop data solutions that support AI, Copilot, analytics and intelligent automation use cases.
- Collaborate with Product Owners, developers, solution engineers and business stakeholders to understand data requirements.
- Implement appropriate data quality, governance, metadata, lineage, privacy and security practices.
- Integrate data pipelines with applications, APIs, Power Platform and automation services.
- Monitor data services and investigate performance, quality and production issues.
- Maintain technical documentation, operational procedures, alerts and support runbooks.
- Identify opportunities to automate, standardise and improve data engineering practices.

About You

You are a skilled Data Engineer who enjoys solving complex problems and creating reliable, scalable data solutions. You bring strong technical capability, sound judgement and the ability to work collaboratively across business, product and technology teams

You will bring:

- Demonstrated experience in data engineering, data integration, analytics engineering or enterprise data delivery.
- Experience designing and supporting data pipelines, transformations, datasets and controlled access solutions.
- Strong knowledge of data engineering, data modelling, database and warehouse design, systems integration and data quality.
- Practical experience with Microsoft Azure data services, SQL, databases, APIs and programming or scripting.
- Understanding of ETL/ELT concepts, automated testing, source control, CI/CD and DevOps practices.
- Knowledge of data governance, privacy, security, access control, metadata and lineage.
- Strong problem-solving, documentation, communication and stakeholder engagement skills.
- Experience supporting AI, automation, analytics or enterprise applications will be highly regarded

This prospect will provide:

- Salary packaging and novated leasing options for eligible employees.
- Company paid parental leave for eligible employees.
- Access to fitness passport.
- Access to Udemy an online training platform with over 5,500 courses.
- Professional development and career progression.
- Experience working with an Iconic Australian Brand with global success in NZ, Asia, and Europe.
- A supportive team environment that celebrates diversity and promotes a healthy work and family life balance.
- Opportunities to make discounted purchases on a great range of products and services.

We’re looking for a Lead Data Engineer to help our clients design, build and operate reliable data platforms that turn complex data into useful business insights.

You’ll work across the data lifecycle, from architecture and source integration through to data pipelines, semantic models, visualisation and reporting. You’ll bring strong engineering skills in Python and SQL, alongside hands‑on experience with at least one of Databricks, Snowflake or Microsoft Fabric.

Working closely with clients, analysts, data scientists and fellow engineers, you’ll translate business requirements into secure, maintainable and scalable data solutions. You’ll also provide technical guidance, contribute to design decisions and help the team strengthen its engineering practices.

Design, build and maintain data pipelines to ingest, transform and deliver data from a range of sources, including APIs.





Develop solutions using Databricks, Snowflake or Microsoft Fabric, selecting approaches that suit the client’s requirements and existing environment.

Contribute to data architecture and solution design, including data storage, integration and modelling.

Develop data models, ontologies and semantic layers that establish consistent business definitions and support analytics and reporting.

Implement pipeline orchestration, scheduling, monitoring and operational support to keep data services reliable.

Troubleshoot data and pipeline issues, improving performance, scalability and cost efficiency.

Apply data quality controls, automated testing and validation throughout the data lifecycle.

Implement role‑based access control and follow security best practices for data platforms and integrations.

Apply DevOps practices to data engineering, including version control, repository management, CI/CD and automated deployment.

Build data visualisations and reports that make information clear and useful to business users.

Document solutions, review code and share knowledge to support maintainable delivery and continuous improvement.

Hands‑on experience delivering production data solutions using one or more of Databricks, Snowflake or Microsoft Fabric.

Strong programming skills in Python and SQL.

Experience designing, developing and operating data pipelines, including ETL/ELT processes, orchestration and monitoring.

A sound understanding of data architecture and modelling techniques, including analytical and dimensional modelling.

An understanding of ontologies and semantic modelling, and how they connect data structures to business concepts.

Practical experience optimising data processing and troubleshooting pipeline, query and platform performance.

Experience integrating data through APIs, including authentication, error handling and reliable data exchange.

Experience applying role‑based access control (RBAC) and data security best practices.

Experience using Git, maintaining repositories, writing automated tests and building CI/CD workflows for data solutions.

Experience developing data visualisations and reports.

The ability to explain technical choices clearly, work collaboratively with business and technical stakeholders, and guide other engineers.

PostgreSQL, SQL Server, Google BigQuery or Amazon Redshift.

R or other programming languages.

Relevant cloud services across Microsoft Azure, AWS or Google Cloud.

Relevant certifications in Databricks, Snowflake or Microsoft Fabric.

Flexible, hybrid working – 20 “work from anywhere” days per year on top of our hybrid policy.

Global career pathways and non‑linear progression; move between domains, disciplines or geographies.

Continuous learning via certifications, internal academies, tech communities and funded conferences.

Well‑being & lifestyle allowance, loyalty leave, inclusive parental leave and employee assistance programmes.

A vibrant, inclusive culture where diverse perspectives power innovation.

Monthly Lifestyle Allowance: Contribution towards health and wellbeing activities like gym memberships.

Novated Leasing: Pre‑tax car leasing benefit for new and used cars.

Loyalty Leave: Receive an additional day of leave on your 3rd, 4th, and 5th work anniversaries, accumulating up to a maximum of 3 extra days of leave per year.

Inclusive Parental Leave Policy: 12 weeks of primary carer’s leave and 4 weeks of secondary carer’s leave.

Work From Anywhere: In addition to our hybrid working policy, we also offer 20 days of working from anywhere per year. Ideal for an extended trip to get away from the city or visiting loved ones.

At Endava, we’re committed to creating an open, inclusive, and respectful setting where everyone feels safe, valued, and empowered to be their best. We welcome applications from people of all backgrounds, experiences, and perspectives—because we know that inclusive teams help us deliver smarter, more innovative solutions for our customers. Hiring decisions are based on merit, skills, qualifications, and potential. If you need adjustments or support during the recruitment process, please let us know.

Every major business function, including Finance, People, Procurement, Tax, and Compliance, operates on its own version of the truth: fragmented, siloed, and manually managed. You’ll build the governed data foundation that replaces this, greenfield,



on a modern stack with real architectural ownership and production delivery in your hands.

Technology is our how, and people are our why. For 20‑plus years we have paired world‑class engineering with deep industry insight, helping global brands create intelligent digital experiences that drive meaningful change. Join us as we expand our Australian Data & AI capability and help clients unlock the full value of their data assets.

We’re looking for a Senior Data Engineer (Databricks, Data Modeling) to model large‑scale batch and streaming data into governed Bronze, Silver and Gold layers to support regulatory reporting data products. You will build on Databricks using Spark, Python and SQL, partner with stakeholders on definitions and use AI‑assisted tools to speed up delivery, testing and documentation.

Design trusted Bronze, Silver and Gold layers using medallion architecture principles.

Model large‑scale batch and streaming datasets into reusable, governed data models for reporting use cases.

Build Databricks pipelines using Delta Lake, Delta Live Tables or Lakeflow aligned to quality and lineage needs.

Develop Spark workloads using PySpark and Spark SQL for performance, reliability and maintainability.

Implement streaming ingestion and processing using Spark Structured Streaming and Kafka or similar.

Partner with business and regulatory stakeholders to translate reporting needs into data products and model definitions.

Strengthen governance with cataloging, lineage and data quality checks using Unity Catalog or equivalent patterns.

Contribute to CI/CD for data pipelines, Git‑based workflows, testing and release practices.

Data modeling background across conceptual, logical and physical models using dimensional, Data Vault or similar approaches.

Databricks experience, including Delta Lake plus governance concepts such as Unity Catalog.

Apache Spark skill sets using PySpark and Spark SQL in production environments.

Streaming data experience using Spark Structured Streaming, Kafka or similar technologies.

Python capability for pipeline development and orchestration patterns.

Advanced SQL skill sets for transformation logic, validation and performance tuning.

Data governance experience covering quality, lineage, controls and well‑defined data contracts.

AI‑assisted development tooling exposure, with Claude preferred, to accelerate build, testing and documentation.

Regulatory reporting data exposure such as EMIR, MiFIR or ASIC derivative reporting.

Data mesh experience delivering domain‑owned data products.

Data Vault 2.0 experience in large‑scale enterprise models.

Apache Kafka experience and strong understanding of streaming platform operations.

Skills & Experience: Familiarity with cloud‑native data engineering patterns and object storage‑based data offload architectures (e.g. AWS S3, Azure Data Lake).

Understanding of real‑time decisioning platforms such as Pega and how they integrate with data engineering layers.

Exposure to MarTech or Transformation data programs in a large enterprise or telco context.

Experience with workflow orchestration tools such as Apache Airflow or equivalent.

Understanding of data governance frameworks, data lineage, and data quality processes.

Knowledge of Agile or SAFe delivery methodology and working in cross‑functional squads.

Design and build Data Model Objects (DMOs), Data Stream Objects (DSOs), and source‑to‑model mapping layers within Salesforce Data Cloud (SFDC360).

Develop and maintain data offload solutions between the Data Cloud and upstream or downstream platforms.

Configure and validate Identity Resolution (IR) rulesets and unified contact profiles within the Data Cloud environment.

Support deployments across development, SIT, UAT, pre‑production, and production environments - including preparation of implementation plans, change management documentation, and post‑go‑live validation.

Manage and govern data onboarding processes including data dictionaries, attribute mapping reviews, and build validation.

3–5+ years of hands‑on data engineering experience in a complex enterprise environment.

Demonstrated experience with Salesforce Data Cloud - including DMO/DSO development, data ingestion, identity resolution, and end‑to‑end deployment lifecycle management.

Strong working knowledge of Databricks, data lake, or similar UDP environment, including batch and streaming ingestion patterns and pipeline orchestration.

Proficiency in SQL and at least one scripting language (Python preferred) for data transformation and pipeline development.

Experience working across multiple delivery environments (Dev / SIT / UAT / Pre‑Production / Production) with formal change management and release governance processes.

Ability to write clear technical documentation - implementation plans, data dictionaries, and solution design notes.

#J-18808-Ljbffr

📌 Senior Platform Reliability Engineer, Ai Data Fabric (Sydney)
🏢 Firmus Technologies
📍 Sydney