Specialist Quality Control Auditor
As a Specialist Quality Control Auditor at Scale AI, I audited MSc-to-PhD-level financial quantitative outputs for accuracy and adherence to AI evaluation standards. My responsibilities included reviewing final AI or contributor-generated outputs for instruction compliance and fact-checking. I also surfaced label inconsistencies in preference-ranking datasets to improve rubric design. • Conducted QC on deep learning model outputs in finance, economics, and accounting domains • Evaluated instruction-following, factual grounding, calculation, and assumption validity • Calibrated reliability on PRBench and preference-ranking datasets for LLM evaluation • Provided feedback to rubric design to ensure edge cases and label consistency