The World as Viewed by 136 Countries: An Open, Curated, Annotated, and Multilingual Dataset of 118 Million State Media Articles
Abstract
Domain fit: AI-core · Core AI workload signals detected from paper context and implementation/artifact evidence.
State media outlets often project their governments' narratives to target both domestic and foreign audiences. Because this content is plentiful and free to access, it appears widely in the web corpora that large language models (LLMs) are trained on and that retrieval systems draw from. However, the prevalence of state media in these corpora and its potential influence on LLM outputs remain largely unexplored. In this paper, we introduce WorldStateMedia, an open dataset of 118 million state media articles from 136 countries in 114 languages. We extract these articles from existing pretraining corpora and annotate each outlet with level of state control from State Media Monitor. Using WorldStateMedia to audit these corpora, we find that, in English web crawls, the share of state media pages from editorially controlled (vs. independent) outlets rose from 39% in 2013 to 93% in 2025. In several low-resource languages, state media make up a substantial share of all web data, such as 17.8% for Uyghur and 7.8% for Belarusian. To characterize the differing narratives of news outlets, we compare Russian and Ukrainian state media in English and in their native languages, revealing asymmetric topical focus and opposing framings of contested issues such as Crimea. Furthermore, in a retrieval-augmented generation (RAG) experiment on territorial disputes, retrieving a country's state media shifts model answers toward that country's claims, most strongly for English-language (foreign-targeted) media. We release WorldStateMedia to support multilingual, cross-national insights on state media at https://huggingface.co/collections/manestay/worldstatemedia
Results and benchmarks
State media outlets often project their governments' narratives to target both domestic and foreign audiences.
Benchmark evidence is limited
Evidence graph: 4 refs, 4 links.
Utility signals: depth 65/100, grounding 85/100, status medium.
Implementation
No direct implementation yet
Maintained implementation evidence is not confirmed for this paper yet.
Use the implementation status and reproduction sections for the current action plan.
No verified maintained repo yet
There is no verified maintained implementation yet. Use this baseline plan to decide whether to prototype now or defer.
- Use the paper-linked Hugging Face release as the starting artifact, then reconstruct training and evaluation settings from the paper.
- No maintained paper-verified implementation was found; start with the closest related repositories below.
- Compare repo methods against the paper equations/algorithm before trusting metrics.
Time to first repro: a few days · Best available artifact: manestay/worldstatemedia
Aryia-Behroziuan/References is the closest maintained adjacent implementation (Matches contextual method/domain keyword: information retrieval). It is not paper-verified; validate algorithm and evaluation setup against the paper before trusting reported metrics. Community adoption signal: 75 GitHub stars.
- Adjacent implementations are not paper-verified
- Recommended repository is adjacent and not paper-verified.
- Adjacent implementation match confidence is low.
Reproduction readiness
No repo
No verified implementation available
- No maintained repository has been identified for this paper. Check adjacent implementations or HF artifacts below.
Hardware requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Validation caveat
Repositories and ecosystem
Closest related implementations
These are not paper-verified. Use them as reference points when no direct implementation is available.
- Aryia-Behroziuan/References Adjacent · Confidence: Low · 75 stars
Matches contextual method/domain keyword: information retrieval
No additional verified repositories beyond the primary recommendation.
Hugging Face artifacts
Models
Broaden model search
Datasets
No trustworthy datasets matches right now.
Search datasets on Hugging FaceSpaces
No trustworthy spaces matches right now.
Search spaces on Hugging FaceResearch context
Tasks
Retrieval / indexing
Methods
Transformer, Retrieval-augmented generation
Domains
Natural Language Processing, Large Language Models, Information Retrieval
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.