HFEPX · Eval paper review
Filippos-Paschalis Rorris, Nikoleta Stanitsa, Tatiana Rorri, Foteini Rorri +1 more
Published
Oct 18, 2026
Citations
0
Trust level
High
Usefulness score
75/100 (High)
Extraction confidence
80% (High)
Derived from extracted protocol signals and abstract evidence.
Rater population
Domain Experts
Signals refreshed
Oct 18, 2026
This paper has strong direct human-feedback and evaluation protocol signal and is suitable as a primary eval pipeline reference.
Use this as a practical starting point for protocol research, then validate against the original paper.
Best use
Primary protocol reference for eval design
Use if you need
A concrete protocol example with enough signal to inform rater workflow design.
What to verify
Validate the exact study setup in the full paper before operational use.
Main weakness
No major weakness surfaced.
Use this as a primary source when designing or comparing eval protocols.
If you are doing eval pipeline work, start here
Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text. We evaluated Morpheus, a fully local de-identification pipeline for Greek clinical documents, on 500 synthetic discharge summaries. Methods: The notes (300 cardiac surgery, 100 cardiology, 100 nephrology) were generated by a frontier large language model (Gemini 3.1 Pro) constrained to authentic Greek hospital document layouts. Notes were pre-annotated at generation; two clinicians then independently reviewed and corrected every note, blinded to each other, with disagreements arbitrated. Primary outcomes were recall over must-redact identifiers and over-redaction of clinical content, measured over must-survive probes. Secondary outcomes: residual leaks flagged by an independent egress-review layer, processing time and reproducibility. Results: Inter-annotator agreement was F1 = 0.998. The de-identification rate was 96.59% (7,666/7,937 identifier instances; 95% CI 96.16-96.96): 98.49% for direct and 93.42% for quasi-identifiers. No patient's own complete identifier survived in any note; of 271 residual instances, 199 (73.4%) concerned institutional identity, chiefly the hospital letterhead. The over-redaction rate was 2.82% (190/6,747 probes; 95% CI 2.45-3.24). The egress guard flagged 2 of 271 residuals, both complete telephone numbers. Two independent runs gave identical verdicts. Conclusions: A fully local, deterministic pipeline removed 98.5% of direct patient identifiers from Greek clinical text while altering under 3% of protected clinical content; most residuals concerned the institution, not the patient. To our knowledge this is the first benchmark evaluation of clinical-text de-identification for Greek and the first quantified evaluation of an independent egress-guard layer.
These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.
Expert Verification
Directly usable for protocol triage.
"Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text."
Automatic Metrics
Includes extracted eval setup.
"Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text."
Inter Annotator Agreement Reported
Calibration/adjudication style controls detected.
"Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text."
Not extracted
No benchmark anchors detected.
"Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text."
F1, Recall, Agreement
Useful for evaluation criteria comparison.
"Notes were pre-annotated at generation; two clinicians then independently reviewed and corrected every note, blinded to each other, with disagreements arbitrated."
Domain Experts
Helpful for staffing comparability.
"Results: Inter-annotator agreement was F1 = 0.998."
No benchmark or dataset names were extracted from the available abstract.
Background and Objective: Clinical free text cannot leave the hospital without de-identification, yet no de-identification system or benchmark exists for Greek clinical text.
Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.
Human feedback protocol is explicit
Detected: Expert Verification
Evaluation mode is explicit
Detected: Automatic Metrics
Quality control reporting appears
Detected: Inter Annotator Agreement Reported
Benchmark or dataset anchors are present
No benchmark/dataset anchor extracted from abstract.
Metric reporting is present
Detected: f1, recall, agreement
