We are looking for an experienced AI Architect with strong expertise in Databricks, Data Engineering, Machine Learning, and Cloud-based AI solutions to lead the design and implementation of scalable AI/ML platforms and intelligent data solutions. The ideal candidate will define enterprise AI architecture, drive up-to-date data and AI transformation initiatives, and work closely with business, engineering, and analytics teams to deliver high-impact solutions.

This role requires a strong understanding of end-to-end AI/ML lifecycle , Lakehouse architecture , Databricks ecosystem , and the ability to translate business use cases into production-ready AI solutions.

Key Responsibilities

- Design and implement enterprise AI/ML architecture using Databricks Lakehouse Platform.
- Define scalable solutions for data ingestion, feature engineering, model training, deployment, monitoring, and governance.
- Architect and optimize AI/ML pipelines using Databricks, Spark, Python, SQL, and cloud-native services.
- Lead the setup of MLOps / LLMOps frameworks for model lifecycle management, CI/CD, model registry, and automated deployment.
- Work with business stakeholders to identify and prioritize AI/ML use cases, including predictive analytics, NLP, recommendation engines, and generative AI.
- Build and guide architecture for LLM / Generative AI solutions, including RAG, vector databases, prompt orchestration, and model integration where applicable.
- Establish best practices for data quality, security, compliance, observability, scalability,



and responsible AI.
- Collaborate with Data Engineers, Data Scientists, Product Owners, and Cloud teams to ensure solution alignment with enterprise architecture standards.
- Provide technical leadership in selecting AI/ML tools, frameworks, and cloud services aligned to business and platform strategy.
- Support architecture reviews, technical design workshops, PoCs, and enterprise AI roadmap planning.

Required Skills & Experience

- 8- 15 years of experience in Data / AI / Analytics architecture, with strong exposure to enterprise-scale implementations.
- Hands-on experience with Databricks including:
- Databricks Lakehouse
- Delta Lake
- Unity Catalog
- MLflow
- Databricks Workflows
- Model Serving / Feature Store (preferred)

Strong programming experience in Python, PySpark, SQL , and AI/ML solution development. Solid experience in designing and deploying Machine Learning pipelines in production. Good understanding of MLOps / LLMOps , including model versioning, CI/CD, deployment, monitoring, and governance. Experience with cloud platforms such as Azure, AWS, or GCP , preferably with Databricks integration. Familiarity with Generative AI / LLM ecosystems , such as OpenAI, Hugging Face, LangChain, vector stores, embeddings, and RAG architectures . Strong understanding of data engineering, ETL/ELT, distributed computing, and data platform modernization. Experience in solution architecture, technical governance, and stakeholder engagement.

#J-18808-Ljbffr

📌 AI Architect - Databricks (City of Sydney)
🏢 Unison Group
📍 City of Sydney