XPT Software Australia Pty Ltd | Contract
Prompt Engineer
Sydney, Australia | Posted on 08/04/2026
XPT SoftwareAustralia PTY Ltd, incorporated in ****, is a Software Services company
XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
We have 120+technocrats in Australia working at our clientlocations.
XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
We have served100+ clients globally, fulfilling their onsite-offshoreneeds.
Job Description
Key Responsibilities
Design,develop, and maintain prompt frameworks, including system prompts,few-shot examples, role-based prompts, and reasoning workflows forproduction-grade LLM applications.
Build andmanage automated evaluation frameworks to measure model performance,accuracy, latency, and regression across releases.
Conductstructured A/B testing across prompt variations, model versions, andconfiguration settings to optimize task-specific outcomes.
Convertproduct requirements and edge-case scenarios into effective promptinstructions, personas, constraints, and guardrails.
Partnerwith ML engineers and product teams to determine when prompt engineeringis sufficient versus when fine-tuning, RAG, or other AI architectures arerequired.
Create andmaintain a centralized prompt repository with version control, documentation,and performance benchmarks for organizational reuse.
Leadred-teaming and adversarial testing exercises to identify jailbreak risks,hallucinations, and model vulnerabilities.
Defineevaluation criteria, annotation guidelines, and quality standards toensure consistency, safety, and reliability of AI-generated outputs.




Mentorengineers and stakeholders on prompt engineering best practices,evaluation methodologies, and the capabilities and limitations of modernLLMs.
Presentprompt strategies, benchmark results, and trade-off analyses to product,engineering, and leadership teams.
Applyadvanced prompting techniques, including chain-of-thought, zero-shot,few-shot, and role-based prompting.
Driveprompt testing, evaluation, benchmarking, and continuous optimizationefforts.
Improve AIresponse quality through systematic assessment, tuning, and refinement.
Managecontext handling and prompt orchestration for complex AI workflows.
Technical Skills
Experiencebuilding automation, evaluation pipelines, APIs, or AI-poweredapplications using enterprise-grade development practices.
Hands-onexperience with LLM platforms, prompt engineering, model evaluation, andAI application development.
Familiaritywith prompt orchestration frameworks, vector databases, RAG architectures,and AI agent workflows.
Understandingof data analysis, experimentation, benchmarking, and performanceoptimization.
Experiencewith version control systems, CI/CD pipelines, and cloud platforms.
Strongknowledge of REST APIs, JSON, and system integration patterns.
Ability tocollaborate effectively with software engineers, data scientists, andproduct teams to deliver production-ready AI solutions.
Experience Requirements
4-7 yearsof combined experience in NLP, AI/ML products, software development,technical writing, or related fields.
At least 2years of direct, hands-on prompt engineering experience with productionLLM applications.
Proventrack record of owning and managing prompt systems end-to-end, from designand implementation through monitoring and optimization in production.
#J-*****-Ljbffr

📌 Prompt Engineer (New South Wales)
🏢 Xpt Software Australia
📍 New South Wales