LLM-assisted Data Extraction and Semantic Cleansing Pipeline
Built an LLM-assisted data extraction pipeline to generate structured recommendation data from Reddit discussions and Amazon product metadata. The workflow included fuzzy matching to resolve sparse explicit product links and semantic cleansing to validate and align outputs. • Scaled data acquisition across 29 subreddits and 6 product categories using URL extraction and ASIN parsing • Applied LLM fuzzy matching when product links were sparse to increase usable samples • Designed semantic cleansing (confirmation, sensitive-word filtering, metadata alignment, and LLM/SBERT validation) • Standardized price mapping and improved metadata coverage while reducing re-run time