Ai judge and annotation expert (Freelance)
Lead annotation, adjudication, and quality assurance for labeled datasets used in NLP and CV projects. Develop and apply annotation guidelines to keep label consistency and reliability across annotators. Perform corrective annotation and analysis to improve dataset quality and usability for model training and evaluation. • Conduct inter-annotator agreement analysis and corrective passes • Refine label taxonomies with researchers and iterate on dataset requirements • Implement QA processes for label consistency and reliability • Use sampling and data checks to validate label quality.