XPT Software Australia Pty Ltd | Contract

Prompt Engineer

Sydney, Australia | Posted on 08/04/2026

- XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company

- XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.

- We have 120+technocrats in Australia working at our clientlocations.

- XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.

- We have served100+ clients globally, fulfilling their onsite-offshoreneeds.

Job Description

Key Responsibilities

- Design,develop, and maintain prompt frameworks, including system prompts,few-shot examples, role-based prompts, and reasoning workflows forproduction-grade LLM applications.

- Build andmanage automated evaluation frameworks to measure model performance,accuracy, latency, and regression across releases.

- Conductstructured A/B testing across prompt variations, model versions, andconfiguration settings to optimize task-specific outcomes.

- Convertproduct requirements and edge-case scenarios into effective promptinstructions, personas, constraints, and guardrails.

- Partnerwith ML engineers and product teams to determine when prompt engineeringis sufficient versus when fine-tuning, RAG, or other AI architectures arerequired.

- Create andmaintain a centralized prompt repository with version control, documentation,and performance benchmarks for organizational reuse.

- Leadred-teaming and adversarial testing exercises to identify jailbreak risks,hallucinations, and model vulnerabilities.

- Defineevaluation criteria, annotation guidelines, and quality standards toensure consistency, safety, and reliability of AI-generated outputs.





- Mentorengineers and stakeholders on prompt engineering best practices,evaluation methodologies, and the capabilities and limitations of modernLLMs.

- Presentprompt strategies, benchmark results, and trade-off analyses to product,engineering, and leadership teams.

- Applyadvanced prompting techniques, including chain-of-thought, zero-shot,few-shot, and role-based prompting.

- Driveprompt testing, evaluation, benchmarking, and continuous optimizationefforts.

- Improve AIresponse quality through systematic assessment, tuning, and refinement.

- Managecontext handling and prompt orchestration for complex AI workflows.

Technical Skills

- Experiencebuilding automation, evaluation pipelines, APIs, or AI-poweredapplications using enterprise-grade development practices.

- Hands-onexperience with LLM platforms, prompt engineering, model evaluation, andAI application development.

- Familiaritywith prompt orchestration frameworks, vector databases, RAG architectures,and AI agent workflows.

- Understandingof data analysis, experimentation, benchmarking, and performanceoptimization.

- Experiencewith version control systems, CI/CD pipelines, and cloud platforms.

- Strongknowledge of REST APIs, JSON, and system integration patterns.

- Ability tocollaborate effectively with software engineers, data scientists, andproduct teams to deliver production-ready AI solutions.

Experience Requirements

- 4-7 yearsof combined experience in NLP, AI/ML products, software development,technical writing, or related fields.

- At least 2years of direct, hands-on prompt engineering experience with productionLLM applications.

- Proventrack record of owning and managing prompt systems end-to-end, from designand implementation through monitoring and optimization in production.

#J-18808-Ljbffr

📌 Prompt Engineer (New South Wales)
🏢 Xpt Software Australia
📍 New South Wales