Lead business analyst for a RAG-based GenAI platform (Lambots) focused on conversational chatbot evaluation and pipeline requirements
Led requirement definition and benchmark evaluation for a RAG-based GenAI chatbot platform used for engineering conversational use cases. Validated LLM outputs for accuracy and relevance by defining test criteria and running domain-expert benchmarks to reduce hallucinations. Monitored response-quality metrics and user engagement to support continuous improvement of retrieval and response generation pipelines. • Defined functional requirements for data ingestion, embedding generation, multimodal indexing, and retrieval pipelines across 100+ data sources • Established benchmark testing with domain experts and reduced hallucinations by 40% • Built executive dashboards to report user satisfaction (85%) and guide product roadmap decisions • Incorporated user feedback and test results to improve model response quality and reduce irrelevant responses