Pradeep K.

Key Skills

Software

AWS SageMaker

Appen

CVAT

Encord

Google Cloud Vertex AI

Labelbox

Roboflow

Clickworker

Mercor

Micro1

Top Subject Matter

LLM evaluation and AI Data Quality & Structured Annotation Specialist (Hindi–English)

Supervised AI dataset annotation and Statistical Model Diagnostics | Python

Multilingual LLM Evaluation Specialist (Hindi–English QA & Reasoning Review)

Top Data Types

Audio

Image

Text

Video

Top Task Types

Computer Programming/Coding

Data Collection

Emotion Recognition

Evaluation/Rating

Object Detection

Polygon

Prompt + Response Writing (SFT)

Question Answering

RLHF

Text Generation

Text Summarization

Transcription

Fine-tuning

Red Teaming

Freelancer Overview

I build evaluation infrastructure for Large Language Models (LLMs) with a focus on reliability, alignment, and silent failure detection. Recently, I designed and implemented a modular LLM evaluation framework that includes: • Multi-criteria scoring architecture (instruction adherence, factual accuracy, coherence, safety, tone) • Dataset validation layer for structural integrity • Statistical performance diagnostics (mean, variance, distribution, failure rate) Annotated and reviewed data used for machine learning and AI training purposes. • Followed detailed project guidelines to ensure labeling accuracy and consistency. • Supported quality control by identifying errors and edge cases in annotated datasets.

Labeling Experience

LLM Safety & Adversarial Evaluation Specialist (Red-Teaming & Alignment)

MercorComputer Code ProgrammingPolygon

Delivered high-accuracy bilingual linguistic support in Hindi and English for AI training and language processing projects. Analyzed spoken and written content to preserve contextual meaning, tone, and linguistic intent across datasets. Applied advanced grammar, syntax, and vocabulary knowledge to ensure language consistency and data quality. Identified ambiguities, dialect variations, and semantic differences critical to improving natural language model performance.

2024 - 2026

LLM Evaluation Engineer | AI Reliability & Alignment Infrastructure | Statistical Model Diagnostics | Python

AppenTextQuestion Answering

Designed and implemented a modular evaluation infrastructure for Large Language Models (LLMs) focused on reliability, alignment, and failure diagnostics. Key components: • Multi-criteria scoring engine (instruction adherence, factual accuracy, coherence, safety, tone) • Dataset validation layer for structural integrity • Statistical diagnostics (mean, variance, distribution, failure rate) • Structured failure taxonomy classification • JSON & CSV export pipeline for reproducible reporting • Clean modular architecture designed for extensibility Built to simulate production-grade evaluation workflows rather than ad-hoc testing. GitHub: [https://github.com/pradeepkumar-ai-byte/llm-evaluation-framework]

2024 - Present

AI Trainer – LLM Evaluation & Calibration (Remote)

Text

Evaluated 500+ LLM responses weekly using structured multi-dimensional rubrics across quality and reliability criteria. Calibrated inter-rater judgments to maintain over 95% agreement consistency through iterative feedback loops. Identified recurring hallucination and reasoning failure patterns to inform downstream model improvement efforts. • Evaluated bilingual Hindi–English outputs for contextual and cultural fidelity. • Collaborated with distributed annotation teams to refine scoring guidelines and reduce ambiguity-related rework by ~20%. • Contributed to failure pattern taxonomying (e.g., hallucination and reasoning breakdown) for evaluation diagnostics. • Performed reliability-oriented review of multi-step responses to detect silent failures and constraint adherence issues.

2023 - Present

Multilingual LLM Evaluation Specialist (Hindi–English QA & Reasoning Review)

AudioPolygon

Performed multilingual evaluation and transcription quality review for AI training datasets. • Assessed linguistic accuracy, grammar, tone, and contextual correctness in English and Hindi language tasks. • Reviewed audio and text data to ensure compliance with quality and formatting guidelines. • Applied consistent judgment across language-focused evaluation workflows supporting AI model improvement.

2024 - 2025

AI Data Quality & Structured Annotation Specialist

ClickworkerImageQuestion Answering

Worked on structured data labeling and AI annotation projects involving image review and evaluation. Tasks included annotating image data, reviewing AI-generated responses, scoring outputs for clarity and correctness, and providing structured, guideline-based feedback to improve language model performance. Work was completed remotely with a focus on accuracy, consistency, and quality standards.

2023 - 2024

Education

Work History