AI Engineer at ShrinQ Consulting Group INC (June 2025–Present)
Developed an LLM observability and validation workflow to assess and improve output reliability prior to delivery. Implemented tracing and evaluation using human-in-the-loop feedback loops to iteratively refine model behavior. Built automated checks to detect low-confidence LLM outputs and reduce risk of incorrect responses. • Used LangSmith for tracing, evaluation, and continuous improvement • Created validation pipelines for low-confidence output detection • Enabled human-in-the-loop feedback to guide improvements • Implemented RAG retrieval enhancements to improve relevance