AI Evaluator / Reviewer Job Description

What are the Key Responsibilities of AI Evaluator / Reviewer

• Evaluate AI model outputs against defined quality, accuracy, safety, and helpfulness criteria.
• Rank and rate model responses to support RLHF and preference modeling workflows.
• Conduct adversarial testing and red-teaming to identify model failure modes and safety risks.
• Document evaluation findings and provide actionable feedback to ML and research teams.
• Apply and refine evaluation rubrics and guidelines for consistency across reviewer pools.
• Participate in calibration sessions to ensure alignment with team evaluation standards.