Who We AreHealthcare needs a better rhythm: one that keeps care continuous and deeply human.
Heidi is building an AI Care Partner that works alongside clinicians to make that possible.We're a team of doctors, engineers, designers, researchers, and creatives building tools that help clinicians stay focused on what matters most: their patients.In just 18 months, Heidi has given back more than 18 million hours to healthcare professionals — supporting 73 million patient visits in 116 countries.
Today, more than two million patient visits each week are powered by Heidi worldwide.Backed by nearly $100 million in funding, we're growing in the US, UK, Canada, and Europe, partnering with leading health systems including the NHS, Beth Israel Lahey Health, and Monash Health.What you'll doLLM platform on KubernetesDesign, deploy and maintain AWS/EKS infrastructure running GPU-backed model workloadsManage GPU node pools, tune autoscaling for inference traffic patterns, and own the full model serving lifecycle from container builds to production rolloutsWrite and maintain infrastructure as code in TerraformModel evaluation and qualityBuild tooling that measures whether models are performing — clinically accurate, latency-appropriate, cost-efficientDesign offline evaluation harnesses, automated regression tests, and dashboards that surface regressions before they reach cliniciansCost and performanceOwn GPU utilization, quantization, request batching, model routing, and spot/on-demand node strategyWork closely with the Models Team on fine-tuning workflows and model selection tradeoffsCross-functional deliveryCollaborate with product engineers and clinicians on prompt engineering, context window management,



and model data pipelinesClose the feedback loop between infrastructure quality and real-world clinical performanceObservabilityInstrument token usage, latency P99s, GPU memory pressure, hallucination rates, and error classesDefine alerting thresholds and build self-serve model health tooling so the team is not relying on Slack threads to know something is wrongWhat we're looking forStrong AWS and Kubernetes experience, with hands-on depth in EKS, GPU workload scheduling, and IAM patterns that do not cut corners on healthcare data requirementsPractical LLMOps experience: model serving frameworks (vLLM, TGI or similar), prompt versioning, model registry management, A/B deployment, shadow traffic, rollback strategiesComfort with Python and enough ML context to hold a real conversation about fine-tuning, RLHF, RAG architectures, and evaluation methodologyInfrastructure-as-code fluency in TerraformExperience building evaluation frameworks for generative models — not just accuracy metrics, but latency, cost, and output safetyA bias toward observable, auditable systems — especially in a regulated industry contextStrong engineering habits: small PRs, meaningful code review, test coverage, and a low tolerance for tech debtWillingness to get close to the clinical domain: you do not need to be a clinician, but you need to care about the consequences of model failures in healthcare BonusExperience working with HIPAA,



the Australian Privacy Act, or similar healthcare data requirementsPrior work on latency-sensitive inference pipelines for consumer or clinical productsFamiliarity with agent orchestration frameworks (LangChain, LangGraph, or similar)Experience with Karpenter or cluster autoscaler tuning for bursty GPU workloadsThe way we work1.
Build to LastWe design for safety and reliability so clinicians, patients, and our teams can trust what we build every day.2.
Own Your PracticeIdeas rise on merit, not title, and everyone shares responsibility for the standards we set together.3.
Move Fast, Stay SteadyWe move quickly but never at the cost of trust.
Progress only matters if people can depend on what we make.4.
Make Others BetterHonest feedback, steady support, and shared growth keep our teams improving together.Why you will flourish with usFlexible hybrid working environment, with 3 days in the office.A generous personal development budget of $500 per annumLearn from some of the best engineers and creatives, joining a diverse teamBecome an owner, with shares (equity) in the company, if Heidi wins, we all winThe rare chance to create a global impact as you immerse yourself in one of Australia's leading healthtech startupsIf you have an impact quickly, the opportunity to fast track your startup career!
Heidi is dedicated to creating an equitable, inclusive, and supportive work environment that brings people together from diverse backgrounds, experiences, and perspectives.
Our strength is in our differences.
We're proud to be an equal prospect employer and welcome all applicants as we're committed to promoting a culture of opportunity for all.

📌 Llm Ops Engineer (Melbourne)
🏢 Heidi
📍 Melbourne