HFEPX · Eval paper review

Does My Embedding Reflect That A = B? Evaluating Mathematical Equivalence in Embedding Models

Jiaying Ye, Samarth Rao, Leo Carlin, Kedar Chintalapati +8 more

Published

Jun 22, 2026

Citations

Trust level

Low

Usefulness score

0/100 (Low)

Extraction confidence

15% (Low)

Derived from extracted protocol signals and abstract evidence.

Rater population

Not reported

Signals refreshed

Jun 26, 2026

Open arXiv Find implementation Shortlist (0) RSS feed

Should you rely on this paper?

This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.

Use this as background context only. Do not make protocol decisions from this page alone.

Abstract-only analysis, low confidence

All signals on this page are inferred from the abstract only and may be inaccurate. Do not use this page as a primary protocol reference.

This paper looks adjacent to evaluation work, but not like a strong protocol reference.
The available metadata is too thin to trust this as a primary source.
The abstract does not clearly name benchmarks or metrics.

Best use

Background context only

Use if you need

A secondary eval reference to pair with stronger protocol papers.

What to verify

Read the full paper before copying any benchmark, metric, or protocol choices.

Main weakness

This paper looks adjacent to evaluation work, but not like a strong protocol reference.

Human feedback signal

Not explicit

Not explicit in abstract metadata

Evaluation signal

Detected

Eval setup described

Usefulness for eval research

0/100

Adjacent candidate

Treat as adjacent context, not a core eval-method reference.

If you are doing eval pipeline work, start here

Human Eval Papers LLM-as-Judge Papers Pairwise Preference Papers Tool-Use Evaluation Search

Abstract

Because mathematics is highly abstract, a single statement can take very different forms depending on what subfield it is framed in. There are many examples where breakthroughs occurred after researchers discovered that a question had already been answered in a different field. At the same time, the growth of new resources related to formalization has increased the need for tools that enable efficient and reliable navigation between mathematical 'languages' (e.g., from Lean to natural language). In this paper, we investigate whether current embedding models capture mathematical equivalence. To do this, we introduce the Mathematically Equivalent but Lexically Different Pairs (MELD) Dataset, a collection of mathematically equivalent statements that are expressed in very different language. We show that current state-of-the-art embedding models tend to group statements by the terminology used to make them instead of the underlying math. Motivated by this, we propose a contrastive approach to learning embeddings of mathematical text that focuses on aligning informal statements with different formalizations. Our experiments demonstrate that this leads to improvements not only on informal-formal retrieval tasks but also on MELD, which only contains natural language statements.

What we could verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.