What is the best open-source implementation of "Evaluating Language Models as Synthetic Data Generators"?

The best maintained implementation is neulab/data-agora with 40 stars on GitHub. Confidence: high. Reproducibility: Limited.

How reproducible is "Evaluating Language Models as Synthetic Data Generators"?

Estimated time to first reproduction: a few days. Risk flags: No CI workflows detected, Dependency manifest is missing. Start with neulab/data-agora and validate setup instructions in README.

Are there pretrained models available for "Evaluating Language Models as Synthetic Data Generators"?

Yes, 3 Hugging Face models found. The top result is MattyB95/AST-ASVspoof5-Synthetic-Voice-Detection with 4,998 downloads.

What framework is used to implement "Evaluating Language Models as Synthetic Data Generators"?

The primary implementation uses pytorch.

Evaluating Language Models as Synthetic Data Generators

Published: Dec 1, 2024

Best maintained implementation now

Evidence: Direct

Domain fit: AI-core

Verified repos: 1

Top repo stars: 40

Core AI workload signals detected from paper context and implementation/artifact evidence.

Framework: pytorch

Time to first repro: a few days

2 risk flags

arXiv PDF

Technical details

Canonical key: arxiv-2412.03679

Cache status: Fresh

Generated at: May 24, 2026, 1:55 PM

Artifact coverage: direct

HF provider: ok (token)

PWC source used: Yes

LLM status: not_generated

LLM model: n/a

LLM generated: Unknown

LLM content type: n/a

HF policy: hf-relevance-v27

implementation starting point

Benchmarks: missing

Time to repro: a few days

2 risk flags

pytorch

Results & Benchmarks

Freshness tier: cold

Direct + Inferred Evidence

No concrete benchmark grounding is available yet. Treat the page as context or an implementation starting point only.

Evaluating Language Models as Synthetic Data Generators is the primary contribution described in this paper.

Use This Implementation Because…

Confidence: high

neulab/data-agora is the strongest maintained implementation based on ranking signals. License is declared (Apache-2.0).

Open neulab/data-agora

Reproduction Risks

No CI workflows detected
Dependency manifest is missing

Hardware Notes

Expect multi-day setup/compute for meaningful reproduction based on current guidance.

Evidence disclosure

Evidence graph: 4 refs, 4 links.

Utility signals: depth 65/100, grounding 85/100, status medium.

Implementation Comparison

Top 1 paths

Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.

neulab/data-agora

best maintained

Maintenance: Stale

Confidence: High

Reproducibility: Limited

Official implementation from Papers with Code · Matched via arXiv identifier search

Stars: 40
Last push: Dec 13, 2024 (527d ago)

Risk flags

No push in 12+ months
No CI pipeline detected
No tagged releases

Best implementation now

neulab/data-agora

Confidence: High

Reproducibility: Limited

[ACL 2025 Main] Official Repository for "Evaluating Language Models as Synthetic Data Generators"

Stars: 40

Forks: 4

Last push: Dec 13, 2024

License: Apache-2.0

Official implementation from Papers with Code

Matched via arXiv identifier search

Strong overlap with paper title keywords

Community adoption signal (40 stars)

License ✓

CI –

Deps –

Docker –

Selected neulab/data-agora as the strongest maintained implementation for new work.
Repository activity is within the last 24 months.

Reproduction readiness

Major Work

Time to first repro: days

Last checked: May 24, 2026

Hardware requirements

Expect multi-day setup/compute for meaningful reproduction based on current guidance.

No dependency manifest — manual reconstruction required

· neulab/data-agora has no requirements.txt, environment.yml, pyproject.toml, or Dockerfile.
· You will need to reverse-engineer dependencies from import statements in the source code.
· Last push was 527 days ago.

Open neulab/data-agora

No benchmark numbers could be verified. You will not be able to validate reproduction correctness against published numbers.