Jiayu S.

Key Skills

Software

Other

Don't disclose

Top Subject Matter

NLP text annotation (intent tagging, entity labeling, consistency QA)

Prompt comparison and model scoring (accuracy/completeness/relevance/safety)

LLM response grading and instruction-following evaluation

Top Data Types

Text

Document

Top Task Types

Classification

RLHF

Freelancer Overview

AI Data Annotation / QA Support Project-based. Core strengths include Other and Don't disclose. Education includes Bachelor’s Degree, N/A. AI-training focus includes data types such as Text and labeling workflows including Classification, RLHF, and Evaluation.

Labeling Experience

Knowledge Data Structuring Independent Practice

OtherText

Organized domain examples into clean, review-friendly formats suitable for downstream model training and evaluation. Emphasized label clarity and traceability so annotations could be reused consistently in later workflows. Applied repeatable quality review standards to ensure structured output grading and dataset readiness. • Entity extraction and label structuring • Traceable, review-friendly data formatting • Repeatable labeling and quality standards • Preparation for downstream training/evaluation

2024 - 2024

Model Output Evaluation Independent Practice

OtherText

Evaluated instruction-following responses by scoring correctness, completeness, clarity, relevance, and policy/safety alignment. Identified missing details, logical errors, and safety concerns across large batches of outputs. Improved consistency by applying structured scoring criteria to maintain stable judgment over repetitive QA cycles. • Instruction-following response quality review • Rubric-based scoring and error identification • Clarity/relevance checks for completeness • Policy and safety alignment assessment

2024 - 2024

Prompt Evaluation / Model Scoring (PLHF) Project-based

Don't discloseTextRLHF

Compared multiple prompts for the same task to identify the strongest and weakest candidate. Provided concise rationale-based scoring of model outputs, considering accuracy, completeness, relevance, and safety. Documented edge cases and added short, actionable review notes to guide prompt and response improvements. • Prompt ranking across candidate prompts • Model output evaluation with rubric-based scoring • Edge case identification and documentation • Safety-aware review for relevance and alignment

2024 - 2024

AI Data Annotation / QA Support Project-based

OtherTextClassification

Annotated text datasets for intent tagging and consistency checking while following task-specific instructions and guidelines. Performed batch quality checks and corrected inconsistent labels to improve annotation accuracy and consistency. Documented and applied sample-level feedback to resolve ambiguity and maintain label reliability across large volumes. • Intent tagging and text classification • Entity labeling and consistency checking • Guideline-based review and QA • Batch-level corrections and feedback incorporation

2024 - 2024

Education

Work History