Bing News Headline and Category News Quality Rating
1. Project Scope Responsible for manual quality control of Microsoft Bing’s top news stories and news across various category channels, including screening high-quality news, filtering out non-compliant and low-quality content, and producing annotated data to optimize news recommendations. 2. Specific Annotation and Quality Control Tasks Top News: Verify the timeliness, authority, and risk factors of candidate news stories, and assign news value ratings; Category News: Assign news articles to the appropriate channel categories, distinguish high-quality news, and flag non-compliant content such as advertisements and misinformation; 3. Project Scale A team of 10 members and 100 freelancers, divided into Top News, Category, and Review groups; average daily review of 20,000 news articles; 19 months of project participation, with a cumulative total of nearly 8.5 million annotated samples. 4. Quality Control Standards Personnel: Pre-employment training + practical assessment required for qualification; new hires undergo a 15-day full-inspection probationary period; Review: 15% of general news articles are randomly selected for re-inspection; 100% of headline news articles undergo re-review; annotation discrepancies are arbitrated by designated personnel; Metrics: General news annotation accuracy ≥98%; headlines ≥99.2%; missed detection rate for non-compliant content below 0.3%; daily review and rectification of mislabeled items.