Basic Text-Processing Automation Script (Python)
Built an automated text-processing workflow to clean and standardize raw text data before downstream labeling or evaluation. Implemented duplicate removal, encoding fixes, and formatting normalization to reduce manual preprocessing effort. Added simple string-matching to flag phrases that may be mistranslated or out of context. • Cleaned and standardized raw text (removed duplicates, fixed encoding, normalized formatting). • Reduced manual cleanup time by ~60% on a 5,000-line sample. • Used string-matching logic to flag potentially mistranslated/out-of-context phrases. • Supported preparation of data for subsequent annotation or quality evaluation tasks.