Skip to content
OpenTrain AIFor AI Companies

HFEPX · Eval paper review

TransLaw: A Large-Scale Dataset and Multi-Agent Benchmark Simulating Professional Translation of Hong Kong Case Law

Xi Xuan, Chunyu Kit

Published

Jul 1, 2025

Citations

0

Trust level

High

Usefulness score

87/100 (High)

Extraction confidence

80% (High)

Derived from extracted protocol signals and abstract evidence.

Rater population

Domain Experts

Signals refreshed

Jun 17, 2026

Open arXiv Find implementation Shortlist (0) RSS feed

Tags

Automatic Metrics Coding Expert Verification Human Eval Law Multi Agent Multilingual

Should you rely on this paper?

This paper has strong direct human-feedback and evaluation protocol signal and is suitable as a primary eval pipeline reference.

Use this as a practical starting point for protocol research, then validate against the original paper.

Best use

Primary protocol reference for eval design

Use if you need

A concrete protocol example with enough signal to inform rater workflow design.

What to verify

Validate the exact study setup in the full paper before operational use.

Main weakness

No major weakness surfaced.

Human feedback signal
Detected
From extracted signals
Evaluation signal
Detected
Eval setup described
Usefulness for eval research
87/100
High-confidence candidate

Use this as a primary source when designing or comparing eval protocols.

If you are doing eval pipeline work, start here

Human Eval Papers LLM-as-Judge Papers Pairwise Preference Papers Tool-Use Evaluation Search

Abstract

Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style. We introduce HKCFA Judgment 97-22, the first large-scale sentence-aligned parallel corpus for HK case law, comprising 344 professionally translated judgments (11,099 sentence pairs; 2.1M tokens) spanning 1997-2022. Building on this resource, we propose TransLaw, a multi-agent framework that decomposes translation into word-level expression, sentence-level translation, and multidimensional review, integrating a specialized Hong Kong legal glossary database, Retrieval-Augmented Generation, and iterative feedback, with four-dimensional expert review covering semantic alignment, terminology, citation, and style. Benchmarking 13 open-source and commercial LLMs, we demonstrate that TransLaw significantly outperforms single-agent baselines across all evaluated models, with convergence within 3 iterations. Human evaluation by 10 certified legal translators using our proposed Legal ACS metric confirms gains in legal-semantic accuracy, while showing that TransLaw still trails human experts in stylistic naturalness. The dataset and benchmark code are available at https://github.com/xuanxixi/TransLaw.

What we could verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.

Human Feedback Types

strong

Expert Verification

Directly usable for protocol triage.

"Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style."

Evaluation Modes

strong

Human Eval, Automatic Metrics

Includes extracted eval setup.

"Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style."

Quality Controls

strong

Adjudication

Calibration/adjudication style controls detected.

"Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style."

Benchmarks / Datasets

missing

Not extracted

No benchmark anchors detected.

"Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style."

Reported Metrics

strong

Accuracy

Useful for evaluation criteria comparison.

"Human evaluation by 10 certified legal translators using our proposed Legal ACS metric confirms gains in legal-semantic accuracy, while showing that TransLaw still trails human experts in stylistic naturalness."

Rater Population

strong

Domain Experts

Helpful for staffing comparability.

"Building on this resource, we propose TransLaw, a multi-agent framework that decomposes translation into word-level expression, sentence-level translation, and multidimensional review, integrating a specialized Hong Kong legal glossary database, Retrieval-Augmented Generation, and iterative feedback, with four-dimensional expert review covering semantic alignment, terminology, citation, and style."

Benchmarks and datasets

No benchmark or dataset names were extracted from the available abstract.

Reported metrics

accuracy
Human feedback details
Uses human feedback
Yes
Feedback types
Expert Verification
Rater population
Domain Experts
Expertise required
Coding, Multilingual
Evaluation details
Evaluation modes
Human Eval, Automatic Metrics
Agentic eval
Multi Agent
Quality controls
Adjudication
Evidence quality
High
Use this page as
Primary protocol reference for eval design

Research brief

Metadata summary

Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style.

Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.

Key takeaways

  • Translating Hong Kong Court Judgments from English to Traditional Chinese is mandated by Articles 8-9 of the Basic Law, yet remains constrained by a shortage of parallel resources and rigorous demands on legal terminology, citation format, and judicial style.
  • We introduce HKCFA Judgment 97-22, the first large-scale sentence-aligned parallel corpus for HK case law, comprising 344 professionally translated judgments (11,099 sentence pairs; 2.1M tokens) spanning 1997-2022.
  • Building on this resource, we propose TransLaw, a multi-agent framework that decomposes translation into word-level expression, sentence-level translation, and multidimensional review, integrating a specialized Hong Kong legal glossary database, Retrieval-Augmented Generation, and iterative feedback, with four-dimensional expert review covering semantic alignment, terminology, citation, and style.

Researcher actions

  • Compare this paper against nearby papers in the same arXiv category before using it for protocol decisions.
  • Validate inferred eval signals (Human evaluation, Automatic metrics) against the full paper.
  • Use related-paper links to find stronger protocol-specific references.

Caveats

  • Generated from abstract + metadata only; no PDF parsing.
  • Signals below are heuristic and may miss details reported outside the abstract.

Recommended queries

Expert verification evaluation Accuracy

Contribution summary

  • We introduce HKCFA Judgment 97-22, the first large-scale sentence-aligned parallel corpus for HK case law, comprising 344 professionally translated judgments (11,099 sentence pairs; 2.1M tokens) spanning 1997-2022.
  • Building on this resource, we propose TransLaw, a multi-agent framework that decomposes translation into word-level expression, sentence-level translation, and multidimensional review, integrating a specialized Hong Kong legal glossary…
  • Benchmarking 13 open-source and commercial LLMs, we demonstrate that TransLaw significantly outperforms single-agent baselines across all evaluated models, with convergence within 3 iterations.

Why it matters for eval

  • Building on this resource, we propose TransLaw, a multi-agent framework that decomposes translation into word-level expression, sentence-level translation, and multidimensional review, integrating a specialized Hong Kong legal glossary…
  • Benchmarking 13 open-source and commercial LLMs, we demonstrate that TransLaw significantly outperforms single-agent baselines across all evaluated models, with convergence within 3 iterations.

Researcher checklist

  • Human feedback protocol is explicit

    Detected: Expert Verification

  • Evaluation mode is explicit

    Detected: Human Eval, Automatic Metrics

  • Quality control reporting appears

    Detected: Adjudication

  • Benchmark or dataset anchors are present

    No benchmark/dataset anchor extracted from abstract.

  • Metric reporting is present

    Detected: accuracy