Marina Sánchez-Torrón, Daria Akselrod, Jason Rauchwerk · Mar 26, 2026 · Citations: 0
Expert Verification Multilingual
← Back to explorer To Write or to Automate Linguistic Prompts, That Is the Question Semantic Alignment across Ancient Egyptian Language Stages via Normalization-Aware Multitask Learning Multilingual KokoroChat: A Multi-LLM Ensemble Translation Method for Creating a Multilingual Counseling Dialogue Dataset Progressive Training for Explainable Citation-Grounded Dialogue: Reducing Hallucination to Zero in English-Hindi LLMs GRACE: A Unified 2D Multi-Robot Path Planning Simulator & Benchmark for Grid, Roadmap, And Continuous Environments Video-Based Reward Modeling for Computer-Use Agents A prospective clinical feasibility study of a conversational diagnostic AI in an ambulatory primary care clinic Gender Bias in MT for a Genderless Language: New Benchmarks for Basque LIT-RAGBench: Benchmarking Generator Capabilities of Large Language Models in Retrieval-Augmented Generation Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTOR EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training BLUFF: Benchmarking the Detection of False and Synthetic Content across 58 Low-Resource Languages Evaluating LLM-Based Translation of a Low-Resource Technical Language: The Medical and Philosophical Greek of Galen Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models ExpLang: Improved Exploration and Exploitation in LLM Reasoning with On-Policy Thinking Language Selection SAMAS: A Spectrum-Guided Multi-Agent System for Achieving Style Fidelity in Literary Translation IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment
Tag: Multilingual
Multilingual evaluation papers that call for domain expertise or specialist review (73 papers).
Papers in tag: 73
Running a Multilingual study?
Post a Job →Research Utility Snapshot
Evaluation Modes
- Automatic Metrics (10)
- Llm As Judge (2)
- Human Eval (1)
Human Feedback Types
- Pairwise Preference (7)
- Expert Verification (3)
- Red Team (3)
Required Expertise
- Multilingual (20)
- Coding (6)
- Medicine (2)
He Huang · Mar 25, 2026 · Citations: 0
Pairwise Preference Automatic Metrics Multilingual
Ryoma Suzuki, Zhiyang Qi, Michimasa Inaba · Mar 24, 2026 · Citations: 0
Pairwise Preference Multilingual
- The quality of ``Multilingual KokoroChat'' was rigorously validated through human preference studies.
- These evaluations confirmed that the translations produced by our ensemble method were preferred from any individual state-of-the-art LLM.
Vedant Pandya · Mar 19, 2026 · Citations: 0
Automatic Metrics Multilingual
Chuanlong Zang, Anna Mannucci, Isabelle Barz, Philipp Schillinger, Florian Lier, Wolfgang Hönig · Mar 11, 2026 · Citations: 0
Simulation Env Multilingual
- Advancing Multi-Agent Pathfinding (MAPF) and Multi-Robot Motion Planning (MRMP) requires platforms that enable transparent, reproducible comparisons across modeling choices.
- We present GRACE, a unified 2D simulator+benchmark that instantiates the same task at multiple abstraction levels (grid, roadmap, continuous) via explicit, reproducible operators and a common evaluation protocol.
Linxin Song, Jieyu Zhang, Huanxin Sheng, Taiwei Shi, Gupta Rahul, Yang Liu · Mar 10, 2026 · Citations: 0
Automatic Metrics Multilingual
- Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction.
- In this work, we study reward modeling from execution video: a sequence of keyframes from an agent trajectory that is independent of the agent's internal reasoning or actions.
Peter Brodeur, Jacob M. Koshy, Anil Palepu, Khaled Saab, Ava Homiar, Roma Ruparel · Mar 9, 2026 · Citations: 0
Expert Verification Automatic Metrics MedicineMultilingual
- Translating these systems into clinical practice requires assessment in real-world workflows with rigorous safety oversight.
- We sought to assess the conversational safety and quality, patient and clinician experience, and clinical reasoning capabilities compared to primary care providers (PCPs).
Amaia Murillo, Olatz-Perez-de-Viñaspre, Naiara Perez · Mar 9, 2026 · Citations: 0
Pairwise Preference Multilingual
- WinoMTeus adapts the WinoMT benchmark to examine how gender-neutral Basque occupations are translated into gendered languages such as Spanish and French.
- FLORES+Gender, in turn, extends the FLORES+ benchmark to assess whether translation quality varies when translating from gendered languages (Spanish and English) into Basque depending on the gender of the referent.
Koki Itai, Shunichi Hasegawa, Yuta Yamamoto, Gouki Minegishi, Masaki Otsuki · Mar 6, 2026 · Citations: 0
Llm As JudgeAutomatic Metrics CodingMultilingual
- To bridge the gap between existing evaluations and practical use, we introduce LIT-RAGBench (the Logic, Integration, Table, Reasoning, and Abstention RAG Generator Benchmark), which defines five categories: Integration, Reasoning, Logic,…
- We use LLM-as-a-Judge for scoring and report category-wise and overall accuracy.
Ajinkya Kulkarni, Sandipana Dowerah, Atharva Kulkarni, Tanel Alumäe, Mathew Magimai Doss · Mar 6, 2026 · Citations: 0
Pairwise Preference Multilingual
- We present RAPTOR, Representation Aware Pairwise-gated Transformer for Out-of-domain Recognition a controlled study of compact SSL backbones from the HuBERT and WavLM within a unified pairwise-gated fusion detector, evaluated across 14…
Aleksei Dorkin, Taido Purason, Emil Kalbaliyev, Hele-Andra Kuulmets, Marii Ojastu, Mark Fišel · Mar 2, 2026 · Citations: 0
Pairwise Preference MathCoding
- We subsequently apply supervised fine-tuning, preference optimization, and chat vector merging to introduce robust instruction-following behavior.
- Evaluation on a comprehensive suite of Estonian benchmarks shows consistent gains in linguistic competence, knowledge, reasoning, translation quality, and instruction-following compared to the original base model and its instruction-tuned…
Jason Lucas, Matt Murtagh-White, Adaku Uchendu, Ali Al-Lawati, Michiharu Yamashita, Dominik Macko · Feb 28, 2026 · Citations: 0
Automatic Metrics CodingMultilingual
- We introduce BLUFF, a comprehensive benchmark for detecting false and synthetic content, spanning 79 languages with over 202K samples, combining human-written fact-checked content (122K+ samples across 57 languages) and LLM-generated…
- We present AXL-CoI (Adversarial Cross-Lingual Agentic Chainof-Interactions), a novel multi-agentic framework for controlled fake/real news generation, paired with mPURIFY, a quality filtering pipeline ensuring dataset integrity.
James L. Zainaldin, Cameron Pattison, Manuela Marai, Jacob Wu, Mark J. Schiefsky · Feb 27, 2026 · Citations: 0
Human EvalAutomatic Metrics Multilingual
- This study presents the first systematic, reference-free human evaluation of large language model (LLM) machine translation (MT) for Ancient Greek (AG) technical prose.
- We assess translation quality using both standard automated evaluation metrics (BLEU, chrF++, METEOR, ROUGE-L, BERTScore, COMET, BLEURT) and expert human evaluation via a modified Multidimensional Quality Metrics (MQM) framework applied to…
Zhicheng Fang, Jingjie Zheng, Chenxu Fu, Wei Xu · Feb 27, 2026 · Citations: 0
Red Team Llm As Judge CodingMultilingual
- Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols.
- We introduce JAILBREAK FOUNDRY (JBF), a system that addresses this gap via a multi-agent workflow to translate jailbreak papers into executable modules for immediate evaluation within a unified harness.
Xun Huang, Simeng Qin, Xiaoshuang Jia, Ranjie Duan, Huanqian Yan, Zhitao Zeng · Feb 26, 2026 · Citations: 0
Red Team Automatic Metrics Multilingual
- Owing to its conciseness and obscurity, classical Chinese can partially bypass existing safety constraints, exposing notable vulnerabilities in LLMs.
- To enhance readability and evaluation accuracy, we further design a classical Chinese to English translation module.
Boqi Chen, Xudong Liu, Jiachuan Peng, Marianne Frey-Marti, Bang Zheng, Kyle Lam · Feb 25, 2026 · Citations: 0
Expert Verification Automatic Metrics MedicineCoding
- Multimodal large language models (MLLMs) have shown great potential in medical applications, yet existing benchmarks inadequately capture real-world clinical complexity.
- We introduce MEDSYN, a multilingual, multimodal benchmark of highly complex clinical cases with up to 7 distinct visual clinical evidence (CE) types per case.
Changjiang Gao, Zixian Huang, Kaichen Yang, Jiajun Chen, Jixing Li, Shujian Huang · Feb 25, 2026 · Citations: 0
Pairwise Preference Multilingual
- Analysis shows that, by enabling on-policy thinking language selection as an action during RL, ExpLang effectively extends the RL exploration space with diversified language preference and improves the RL exploitation outcome with leveraged…
Jingzhuo Wu, Jiajun Zhang, Keyan Jin, Dehua Ma, Junbo Wang · Feb 23, 2026 · Citations: 0
Automatic Metrics Multilingual
- To address this, we introduce the Style-Adaptive Multi-Agent System (SAMAS), a novel framework that treats style preservation as a signal processing task.
- Extensive experiments on translation benchmarks show that SAMAS achieves competitive semantic accuracy against strong baselines, primarily by leveraging its statistically significant advantage in style fidelity.
Priyaranjan Pattnayak, Sanchari Chowdhuri · Feb 18, 2026 · Citations: 0
Red Team CodingMultilingual
- Safety alignment of large language models (LLMs) is mostly evaluated in English and contract-bound, leaving multilingual vulnerabilities understudied.
- We introduce Indic Jailbreak Robustness (IJR), a judge-free benchmark for adversarial safety across 12 Indic and South Asian languages (2.1 Billion speakers), covering 45216 prompts in JSON (contract-bound) and Free (naturalistic) tracks.
Yuyan Bu, Xiaohao Liu, ZhaoXing Ren, Yaodong Yang, Juntao Dai · Feb 18, 2026 · Citations: 0
Pairwise Preference Multilingual
- The widespread deployment of large language models (LLMs) across linguistic communities necessitates reliable multilingual safety alignment.
- In this work, we propose a resource-efficient method for improving multilingual safety alignment.