Note: The job is a remote job and is open to candidates in USA. Skill is a tech real estate company seeking licensed real estate agents located in Seattle, WA to work part-time as Agentic AI Judges. The role involves reviewing AI/ML processes related to verbal interactions between home buyers and agents, ensuring quality standards, and collaborating with engineers and data scientists to optimize model performance.
Responsibilities
- Review and/or evaluate annotated data to ensure it meets quality standards and project requirements
- Provide feedback and guidance to engineers to improve accuracy
- Collaborate with engineers and data scientists to optimize model performance based on annotated data
- Develop and maintain annotation guidelines and best practices
- Provide training data to engineers to establish Ground Truth data
- Provide input/examples on Training content
- Identify gaps in training content and suggest improvements
Skills
- Must have an active Real Estate License, or previously been licensed with experience selling real estate in Seattle
- Minimum of 3+ years of active experience as a licensed real estate agent or broker, demonstrating a deep understanding of the real estate transaction process
- Excellent attention to detail and analytical skills
- Ability to work effectively in a collaborative team environment
- Demonstrated deep interest and curiosity in Artificial Intelligence, including experience reviewing or evaluating AI-generated content
- Experience with copywriting, User Experience authoring, and content development around voice and brand guidelines
- Experience with Machine-generated Speech evaluation, Search Results Evaluation, Search Query Categorization
- Experience following or managing complex processes, and working both independently and with a team to achieve a goal
- Experience with tools and software used for Natural Language Processing annotation is a plus
- Experience working on Natural Language Processing projects with a focus on transcriptions, localization, and translation
- Bachelor's degree in English, Linguistics or a related field
Company Overview
Skill is a recruiting technology company that operates an integrated, full-service staffing business. It was founded in 2025, and is headquartered in Boston, Massachusetts, US, with a workforce of 501-1000 employees. Its website is https://www.skill.com.