We are looking for a Senior Systems Test Engineer to take the lead on quality across the agents we build and the systems they connect to. This is a hands‑on job. You will decide how we test, do the harder testing yourself, and your call is the one that decides whether an agent or a release goes live with a client. You will report to the Director and work with our developers, AI engineers and client teams from discovery through to post‑deployment review. You will also be the person our less experienced staff come to.

Key responsibilities

Own the test plan for every agent build, release and system change, deciding what gets tested and what does not, working out which environments and test data you need, setting the entry and exit criteria, and giving the project a realistic estimate of how long testing will take

Go through discovery notes, process maps, business requirements and agent design specifications early, before anyone writes code or a prompt, and build the mapping from requirements to test cases

Write the test conditions, test cases and test scripts used across projects, including invalid input, boundary values, negative cases, ambiguous customer requests and error handling

Plan and run functional, business process, system, integration and regression testing across the agents and the platforms they connect to, including booking, point‑of‑sale, CRM, practice management and document systems

Find, reproduce and diagnose faults in systems and application code, working to testing protocols, guidelines and quality standards

Build and maintain automated tests, knowing which checks are worth automating, keeping scripts current as agents, prompts, APIs and interfaces change

Keep the regression suite current by adding scenarios from defects, updating or dropping cases that no longer match the system

Set up and manage test environments and test data, including sandbox client systems and de‑identified data sets

Plan and run non‑functional testing, including load, response time, cost per interaction, failover and recovery scenarios

Design and run evaluation suites for AI agents with realistic scenarios drawn from each client’s real workflows,



with transparent pass criteria for task completion, accuracy, tone and correct escalation

Test how agents plan and act across multiple steps, whether they choose the right tool, call it with the right parameters, handle failed or slow API responses, recover from errors and stop when they should

Check agent output for factual accuracy and grounding, confirming that answers, summaries and drafted documents are supported by the client’s own source material

Run adversarial and safety testing, including prompt injection, attempts to extract confidential or personal information, off‑topic and abusive inputs, and guardrail bypass attempts

Coordinate User Acceptance Testing with clients and support them while they run it, carry out pre‑change validation, cutover checks and post‑implementation testing

Review and assess the systems architecture behind each agent, looking at how a proposed change will land on the client’s interfacing systems

Produce the test summary and defect reporting that management and clients use at each gate

Create and maintain technical documentation for agents and environments, including test procedures, evaluation sets, technical reports and specifications

Look at test results, evaluation scores and defect trends over time to pick up recurring faults and put forward improvements to the way we test and release

About you

Bachelor degree or higher in Information Technology, Computer Science, Software Engineering or something close to it, or at least three years of relevant work experience

At least three years of hands‑on systems or software testing, including having owned the testing for a release or a project from start to finish

Real depth across functional, integration, system,



regression and user acceptance testing, in a place where several applications talk to each other

Comfortable in test management and defect tracking tools such as Jira, Azure DevOps, TestRail or similar, and keep traceability from requirement through to result

Have built or maintained automated tests, and can tell which ones are worth automating and which are not

Experience testing APIs and integrations, using tools such as Postman or scripted tests, and enough SQL to check data yourself

Exposure to testing AI or machine learning features, such as chatbots, LLM‑based applications or AI agents, including evaluation design, prompt injection or output quality checks

Some scripting ability, ideally Python, and some exposure to load, performance and failover testing

Working knowledge of cloud platforms, web applications, networking basics and common business systems

Strong fault finding, methodical, evidence led, and not satisfied until you know the actual cause

Write clearly, with documentation and test reports usable by someone who is not technical, including clients

Full and unrestricted working rights in Australia

About us

Monster Hub is an Australian AI agency based in Spring Hill, Brisbane. We build custom AI agents for hospitality and professional services businesses: restaurants, law firms, accountants and financial advisers. Our agents take customer enquiries and bookings, manage orders, support rostering, onboard clients, review and summarise documents, track deadlines and prepare reports. Every one of them is built around a real client’s operation and plugged into the systems that client already runs, so when an agent gets something wrong, a customer or a client notices it.

Permanent, full time, 38 hours a week, Monday to Friday

Real ownership of quality across AI agents running in live businesses, with the authority that goes with it

Hands‑on experience in agentic AI testing, one of the fastest growing areas of software quality

Support for training and certification

Spring Hill office, close to Brisbane CBD and public transport

#J-18808-Ljbffr

📌 Senior Systems Test Engineer (Brisbane City)
🏢 Monster Hub pty
📍 Brisbane City