Lead AI Training Specialist (Contractor) — AI Training and LLM Evaluation (Remote)
Works as an AI Training Specialist evaluating and improving LLM outputs using RLHF methods and related training frameworks. Performs prompt-and-response assessment for multi-turn conversations, focusing on factual accuracy, reasoning coherence, and constraint adherence. Produces structured justifications and truthfulness ratings to support model quality improvement and safety. • Evaluate, rank, and rewrite AI-generated responses for few-shot/multi-turn dialogues. • Identify hallucinations, logical fallacies, and formatting errors with detailed justifications. • Support adversarial red-teaming to reduce toxic or unauthorized outputs by ~15% across test batches. • Develop and benchmark developer-focused LLM behaviors via complex coding prompt tasks.