Abhay G.

Key Skills

Software

No software listed

Top Subject Matter

Software Engineering – Code Evaluation & Benchmarking

Artificial Intelligence – LLM Evaluation & Testing

Technology – Developer Tools & Automation

Top Data Types

Computer Code Programming

Top Task Types

Computer Programming/Coding

Freelancer Overview

I've spent the last few months doing this exact work. At Shipd (Datacurve, YC), I authored SWE-bench-style benchmarks against real open-source repositories — reading production codebases, identifying genuine bugs and missing features, writing deterministic test suites, building reproducible Docker environments. The full cycle, not just one piece. The part that actually took time was learning to read unfamiliar codebases quickly and find things worth turning into problems. Most repositories don't advertise their bugs. You read the code, form a hypothesis, design tests that verify the fix without being trivially gameable, and iterate. I have a decent number of approved submissions from that process

Labeling Experience

AI Coding Benchmark Contributor

TextEvaluation/Rating

Worked on SWE-bench-style AI coding benchmark creation using large open-source repositories. Designed issue specifications, deterministic test suites, reference patches, and reproducible Docker environments for evaluating software engineering agents. Analyzed production-scale codebases, validated multi-file implementations, and developed behavioral evaluation workflows to ensure reliable benchmark execution and regression-free testing across containerized environments.

2025 - 2026

Education

Work History