Job Description

We are looking for an experiencedAI Architect with strong expertise inDatabricks, Data Engineering, Machine Learning, and Cloud-based AI solutions to lead the design and implementation of scalable AI/ML platforms and intelligent data solutions. The ideal candidate will define enterprise AI architecture, drive modern data and AI transformation initiatives, and work closely with business, engineering, and analytics teams to deliver high-impact solutions.

This role requires a strong understanding ofend-to-end AI/ML lifecycle ,Lakehouse architecture ,Databricks ecosystem , and the ability to translate business use cases into production-ready AI solutions.

Key Responsibilities

- Design and implemententerprise AI/ML architecture usingDatabricks Lakehouse Platform .
- Define scalable solutions fordata ingestion, feature engineering, model training, deployment, monitoring, and governance .
- Architect and optimizeAI/ML pipelines usingDatabricks, Spark, Python, SQL, and cloud-native services .
- Lead the setup ofMLOps / LLMOps frameworks for model lifecycle management, CI/CD, model registry, and automated deployment.
- Work with business stakeholders to identify and prioritizeAI/ML use cases , including predictive analytics, NLP, recommendation engines, and generative AI.
- Build and guide architecture forLLM / Generative AI solutions , includingRAG, vector databases, prompt orchestration, and model integration where applicable.
- Establish best practices fordata quality, security, compliance, observability, scalability,



and responsible AI .
- Collaborate with Data Engineers, Data Scientists, Product Owners, and Cloud teams to ensure solution alignment with enterprise architecture standards.
- Provide technical leadership in selecting AI/ML tools, frameworks, and cloud services aligned to business and platform strategy.
- Support architecture reviews, technical design workshops, PoCs, and enterprise AI roadmap planning.

Required Skills & Experience

- 8- 15 years of experience inData / AI / Analytics architecture , with solid exposure to enterprise-scale implementations.
- Hands‑on experience withDatabricks including:
- Databricks Lakehouse
- Delta Lake
- Unity Catalog
- MLflow
- Databricks Workflows
- Model Serving / Feature Store (preferred)

- Strong programming experience inPython, PySpark, SQL , and AI/ML solution development.
- Solid experience in designing and deployingMachine Learning pipelines in production.
- Good understanding ofMLOps / LLMOps , including model versioning, CI/CD, deployment, monitoring, and governance.
- Experience withcloud platforms such asAzure, AWS, or GCP , preferably with Databricks integration.
- Familiarity withGenerative AI / LLM ecosystems , such asOpenAI, Hugging Face, LangChain, vector stores, embeddings, and RAG architectures .
- Strong understanding ofdata engineering, ETL/ELT, distributed computing, and data platform modernization .
- Experience insolution architecture, technical governance, and stakeholder engagement .

#J-18808-Ljbffr

📌 AI Architect - Databricks (City of Parramatta Council)
🏢 Unison Group
📍 City of Parramatta Council