Human Feedback Types
strongRubric Rating
Directly usable for protocol triage.
"As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense."
HFEPX · Eval paper review
PhyRubric: Rubric-Grounded Diagnostic Evaluation of Physical Commonsense in Text-to-Video Generation
Huiyi Chen, Weitai Kang, Shicheng Fan, Xinshuo Wang +4 more
Published
Nov 10, 2026
Citations
0
Trust level
Moderate
Usefulness score
57/100 (Medium)
Extraction confidence
65% (Moderate)
Derived from extracted protocol signals and abstract evidence.
Rater population
Not reported
Signals refreshed
Nov 10, 2026
Should you rely on this paper?
This paper has useful evaluation signal, but protocol completeness is partial; pair it with related papers before deciding implementation strategy.
Use this for comparison and orientation, not as your only source.
Low-signal caution for protocol decisions
Use this page for context, then validate protocol choices against stronger HFEPX references before implementation decisions.
- The abstract does not clearly name benchmarks or metrics.
Best use
Secondary protocol comparison source
Use if you need
A secondary eval reference to pair with stronger protocol papers.
What to verify
Read the full paper before copying any benchmark, metric, or protocol choices.
Main weakness
The abstract does not clearly name benchmarks or metrics.
Human feedback signal
Detected
From extracted signals
Evaluation signal
Detected
Eval setup described
Usefulness for eval research
57/100
Moderate-confidence candidate
Useful as a secondary reference; validate protocol details against neighboring papers.
If you are doing eval pipeline work, start here
Abstract
As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense. However, existing benchmarks rely on annotators who rate each video with a single score based on intuition, yielding ratings that are subjective and do not tell which physical principle is violated. We introduce PhyRubric, a rubric-grounded diagnostic framework for evaluating physical commonsense in T2V generation. For each caption, PhyRubric constructs a caption-specific rubric before any video is seen, decomposing the caption into semantic and physical items, and derives scores from item-level verdicts grounded in timestamped evidence. We instantiate this framework as PhyRubric-Bench, which spans 34 categories across five physical domains, and scale it up to PhyRubric-Train with 3,788 videos, on which we fine-tune the open-source PhyRubric-Judge. Empirical results across ten T2V models show that physical commonsense remains far from solved, and that PhyRubric-Judge correlates more closely with human judgments than existing open-source judges while matching the frontier proprietary Gemini 4 Argon. Further analysis provides actionable insights for T2V models.
What we could verify
These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.
Evaluation Modes
strongSimulation Env
Includes extracted eval setup.
"As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense."
Quality Controls
missingNot reported
No explicit QC controls found.
"As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense."
Benchmarks / Datasets
missingNot extracted
No benchmark anchors detected.
"As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense."
Reported Metrics
missingNot extracted
No metric anchors detected.
"As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense."
Benchmarks and datasets
No benchmark or dataset names were extracted from the available abstract.
Reported metrics
No metric terms were extracted from the available abstract.
Human feedback details
- Uses human feedback
- Yes
- Feedback types
- Rubric Rating
- Rater population
- Not reported
- Unit of annotation
- Multi Dim Rubric
- Expertise required
- Medicine
Evaluation details
- Evaluation modes
- Simulation Env
- Agentic eval
- None
- Quality controls
- Not reported
- Evidence quality
- Moderate
- Use this page as
- Secondary protocol comparison source
Research brief
Metadata summary
As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense.
Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.
Key takeaways
- As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense.
- However, existing benchmarks rely on annotators who rate each video with a single score based on intuition, yielding ratings that are subjective and do not tell which physical principle is violated.
- We introduce PhyRubric, a rubric-grounded diagnostic framework for evaluating physical commonsense in T2V generation.
Researcher actions
- Compare this paper against nearby papers in the same arXiv category before using it for protocol decisions.
- Check the full text for explicit evaluation design choices (raters, protocol, and metrics).
- Use related-paper links to find stronger protocol-specific references.
Caveats
- Generated from abstract + metadata only; no PDF parsing.
- Signals below are heuristic and may miss details reported outside the abstract.
Recommended queries
Contribution summary
- As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense.
- However, existing benchmarks rely on annotators who rate each video with a single score based on intuition, yielding ratings that are subjective and do not tell which physical principle is violated.
Why it matters for eval
- We introduce PhyRubric, a rubric-grounded diagnostic framework for evaluating physical commonsense in T2V generation.
Researcher checklist
-
Human feedback protocol is explicit
Detected: Rubric Rating
-
Evaluation mode is explicit
Detected: Simulation Env
-
Quality control reporting appears
No calibration/adjudication/IAA control explicitly detected.
-
Benchmark or dataset anchors are present
No benchmark/dataset anchor extracted from abstract.
-
Metric reporting is present
No metric terms extracted.