Design, develop, and maintain scalable and robust machine learning pipelines on Azure Databricks.

Implement and optimize ML workflows, including data ingestion, feature engineering, model training, evaluation, and deployment.

Utilize Azure Machine Learning (Azure ML) services for model management, experimentation, and MLOps best practices (e.g., MLflow).

Develop and deploy Large Language Models (LLMs) and Generative AI (GenAI) solutions on Databricks, including RAG architectures and fine-tuning.

Work with PySpark, Python, and SQL for data processing, analysis, and model development.

Monitor and troubleshoot deployed models, ensuring data quality, model performance, and detecting drift.

Required Skills & Qualifications:

2+ years of hands-on experience in AI/ML engineering,



with a robust focus on MLOps and cloud-based deployments.

Proven expertise with Azure Databricks and its ecosystem (Delta Lake, MLflow, Unity Catalog).

Proficiency in Python (including PySpark) and SQL.

Experience with Azure Machine Learning services.

Strong understanding of machine learning algorithms, deep learning concepts, and statistical analysis.

Experience with building and deploying Generative AI/LLM solutions is highly desirable.

Excellent problem-solving, communication, and collaboration skills.

#J-18808-Ljbffr

📌 AI/ML Engineer (Azure & Databricks) (City of Sydney)
🏢 RESOURCE CORNER
📍 City of Sydney