Jonathan R.

Key Skills

Software

Mercor

Mindrift

OneForma

Clickworker

Appen

Other

Top Subject Matter

LLM evaluation and human preference data for training/assessment

Automated LLM evaluation and structured scoring for training optimization

Synthetic data collection for multimodal AI training (GIS, simulation, and DICOM-based datasets)

Top Data Types

Video

Image

Document

Text

Top Task Types

Text Generation

Question Answering

Text Summarization

Transcription

Evaluation/Rating

Data Collection

Prompt + Response Writing (SFT)

RLHF

Freelancer Overview

Full-Stack Software Engineer, Outlier AI (2024 – Present). Brings 5+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Internal, Proprietary Tooling, and Other. Education includes Doctor of Philosophy, Massachusetts Institute of Technology (MIT) (2026) and Master of Science, Massachusetts Institute of Technology (MIT) (2023). AI-training focus includes data types such as Text, Geospatial, and Tiled Imagery and labeling workflows including Evaluation, Rating, and RLHF.

Labeling Experience

Senior Full-Stack / AI Engineer, Handshake AI (2025 – Present)

TextRLHF

Architected distributed services for automated AI evaluation pipelines that produced model scoring signals for downstream training optimization. Designed structured model evaluation frameworks to improve LLM output accuracy, prompt durability, and security alignment. Integrated asynchronous, low-latency API and queued background processing to support repeated evaluation runs at scale. • Built FastAPI/Next.js/Redis-backed evaluation APIs • Engineered Redis-backed queues to decouple and accelerate heavy evaluation jobs • Developed structured model evaluation frameworks for reliability and alignment • Improved evaluation output accuracy and security alignment through prompt and framework iteration

2025 - Present

Full-Stack Software Engineer, Outlier AI (2024 – Present)

Text

Built and operationalized evaluation workflows for LLM training outputs using interactive human preference data collection. Implemented programmatic evaluation annotations and tracked grading trends to support dataset and model assessment cycles. Led analysis and benchmarking to isolate performance issues affecting evaluation readiness. • Created reactive internal dashboards for visualization of LLM training outputs • Designed database schemas and automated workflows for evaluation annotation tracking • Optimized SQL indexing and query structures for large volumes of evaluation records • Profiled and benchmarked code to remediate memory leaks and performance regressions

2024 - Present

AI Training & Technical Project Specialist, Freelance Contractor (2023 – Present)

OtherData Collection

Engineered high-fidelity multimodal dataset projects to support training of models using spatial, physical, and mechanical simulation. Built GIS mapping pipelines and terrain analysis datasets using QGIS-based spatial overlays. Produced reconstructed 3D medical datasets from DICOM sources to expand training coverage across biomedical modalities. • Designed GIS mapping pipelines and advanced terrain analysis projects • Developed scientific visualization workflows (ParaView) for reconstructed 3D medical datasets • Created parametric CAD prototypes and mechanical assemblies for simulation-based training data • Authored verification-oriented technical documentation and step-by-step dataset workflow guides

2023 - Present

Machine Learning Engineer, IBM (2022 – 2023)

Delivered NLP service infrastructure and data preparation capabilities that supported training pipelines and automated release validation. Managed enterprise-scale databases used to produce refined assets for training pipelines. Implemented CI/CD automation to run linting, unit tests, and deployment for NLP services to ensure training workloads were stable and reproducible. • Scaled ML model serving microservices with low-latency guarantees • Integrated CI/CD routines for automated quality checks of NLP services • Cleaned, transformed, and managed databases providing refined training assets • Supported reliable pipeline operations through automated testing and deployment workflows

2022 - 2023

Education

Work History