LLM Output Evaluation & JSON Quality Rating
Independent AI training focused on LLM output evaluation and error correction for structured data tasks. Scope: Reviewed AI-generated JSON drafts against source legal documents and strict schemas. Rated outputs for accuracy, schema compliance, and data integrity. Key evaluation tasks: - Compared AI JSON to source PDFs: identified missing entities, wrong dates, type errors - Rated outputs on criteria: field accuracy, null-handling, case sensitivity, number formatting - Identified common LLM errors: DD/MM vs MM/DD dates, string/number type mismatches, hallucinated fields - Rewrote incorrect AI outputs to produce gold-standard JSON for SFT datasets - Provided error categories and correction guidelines for model improvement Quality: Applied 15-point validation checklist covering syntax, schema, and factual accuracy. Tools: JSONLint, diff checkers, schema validators, annotation guidelines.