The Automatic Verification of Image-Text Claims (AVerImaTeC) Shared Task

Research Summary

Contribution Summary

• The Automatic Verification of Image-Text Claims (AVerImaTeC) shared task aims to advance system development for retrieving evidence and verifying real-world image-text claims.
• Participants were allowed to either employ external knowledge sources, such as web search engines, or leverage the curated knowledge store provided by the organizers.
• System performance was evaluated using the AVerImaTeC score, defined as a conditional verdict accuracy in which a verdict is considered correct only when the associated evidence score exceeds a predefined threshold.

Why It Matters For Eval

• The winning team, HUMANE, achieved an AVerImaTeC score of 0.5455.
• This paper provides a detailed description of the shared task, presents the complete evaluation results, and discusses key insights and lessons learned.

Related Papers

• Language Modeling and Understanding Through Paraphrase Generation and Detection Embedding Neighbor
• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation Embedding Neighbor
• Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Framework Embedding Neighbor
• BankMathBench: A Benchmark for Numerical Reasoning in Banking Scenarios Embedding Neighbor
• Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning Embedding Neighbor
• Seeing Through the Noise: Improving Infrared Small Target Detection and Segmentation from Noise Suppression Perspective Embedding Neighbor
• Agentic Adversarial QA for Improving Domain-Specific LLMs Embedding Neighbor
• MixSarc: A Bangla-English Code-Mixed Corpus for Implicit Meaning Identification Embedding Neighbor
• Mind the Style: Impact of Communication Style on Human-Chatbot Interaction Tag Overlap
• Modeling Distinct Human Interaction in Web Agents Tag Overlap
• Onboard-Targeted Segmentation of Straylight in Space Camera Sensors Tag Overlap
• Usability Study of Security Features in Programmable Logic Controllers Tag Overlap