TestMate graduation project (LLM fine-tuning and evaluation for test scaffolding)
Mohammed developed an autonomous agentic testing framework that combines graph-based context extraction with LLM-driven semantic reasoning. He fine-tuned an open-source code model using 4-bit quantized LoRA/PEFT and trained with a rejection sampling pipeline on custom Python datasets. The system also used post-run auditing to evaluate generated behaviors and isolate real defects from LLM hallucinations. • Fine-tuned Qwen2.5-Coder-7B with 4-bit LoRA/PEFT • Applied rejection sampling to improve training outcomes • Built a three-gate post-run audit pipeline (syntax validation, pytest flakiness detection, behavioral incoherence mapping) • Evaluated performance on a benchmark using mutation score and line coverage