AI Data Trainer & Content Quality Analyst (Remote), Appen
Evaluate and rate AI model outputs for accuracy, tone, coherence, and contextual appropriateness to support RLHF pipelines. Annotate and classify large-scale linguistic datasets for NLP model training and fine-tuning. Perform prompt engineering evaluations using semantic analysis to assess intent and improve instruction-following and response quality. • Accuracy and quality rating of LLM outputs • Prompt + response evaluation and flagging against quality benchmarks • Semantic analysis with intent/language pattern labeling • Guideline refinement with cross-functional teams