Mercor is partnering with a leading legal AI company to benchmark how well AI systems answer real questions of Australian law. We are hiring experienced in-house counsel to serve as expert reviewers — running the same prompts through two AI platforms, comparing the answers, and scoring them against a standardized rubric. — What you'll do
- Run the same set of prompts across two AI (LLM) platforms
- Compare the outputs from both platforms side by side
- Score each response against a standardized rubric we provide
- Submit concise written feedback for every evaluation — Requirements
- 5+ years as in-house legal counsel practising in Australia
- Strong command of Australian law, especially commercial and contract matters
- Sharp attention to detail and accurate written English
- Able to work independently to a rubric and to deadlines — Logistics
- Initial commitment: ~10 hours, with strong potential for more
- Confidentiality: all work is covered by NDA
Relevant skills
AI evaluation
📌 In-House Counsel (Australia) (City of Perth)
🏢 HumaniTapp
📍 City of Perth
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.