RADICAL TRANSPARENCY

We publish exactly how we score every article — every trigger term, every threshold, every prompt. If you think we got it wrong, tell us.

5143

ARTICLES (7 DAYS)

828

AI ANALYZED

4015

GEOLOCATED

SOURCES MONITORED

BIAS SCORING METHODOLOGY

Three-Signal System

Every article receives up to three independent bias signals. The ensemble average is the displayed band — more signals = more defensible measurement.

Signal 1 — Heuristic (v1/v3): Keyword frequency analysis against a weighted trigger-phrase list. Fast, runs on every article at ingest.
Signal 2 — LLM Tier-1: B.I.A.S. (qwen2.5:7b via DHQ) reads title + body and returns a structured score with rationale and trigger phrases. Uses Canadian political context (Liberal/NDP/Conservative spectrum). Runs on priority articles.
Signal 3 — ML v2 (DistilBERT zero-shot): valhalla/distilbart-mnli-12-3 runs NLI classification against left/center/right labels. CPU-only, independent of the LLM path. Backlog worker scores ~50 articles per 30-min cycle. US-trained — Canadian political framing (NDP/Liberal/Conservative) may not map cleanly; treated as a second opinion, not ground truth.
Ensemble: 0.5 × heuristic_score + 0.5 × ml_v2_score (where both are available). Band derived from same thresholds as single-signal scoring.

Bias Band Thresholds

LEFTscore < −0.6 — NDP, labour, progressive

CENTER-LEFT−0.6 ≤ score < −0.2 — Liberal, CBC framing

CENTER−0.2 ≤ score < 0.2 — neutral/factual reporting

CENTER-RIGHT0.2 ≤ score < 0.6 — fiscal conservatism, business

RIGHTscore ≥ 0.6 — Conservative base, Postmedia editorial

F.A.C.T. Priority Scoring

F.A.C.T. (gemma3:12b) assigns a priority score to each article each cycle. Articles scoring ≥ 1 join the AI analysis queue (up to 20 per cycle):

score += 3 # same story in 3+ outlets score += 2 # heuristic bias |score| > 0.5 score += 1 # breaking keyword in title

FACT-CHECK METHODOLOGY

V.E.R.I.F.Y. Claim Extraction

V.E.R.I.F.Y. (gemma2:9b) reads priority articles and extracts verifiable factual claims. Each claim receives a verdict:

verified — claim confirmed by public record or widely accepted source
mixed — claim is partly true or requires context
disputed — claim contradicts known evidence
unverifiable — specific claim exists but cannot be checked with available evidence
descriptive_reporting — article contains no specific factual assertions (market summaries, sports scores, breaking dispatches). This is not a quality judgment.

V.E.R.I.F.Y. does not have internet access. Verdicts are based on the model's training data (cutoff 2024). We flag this limitation prominently on every article.

GEOLOCATION METHODOLOGY

L.O.C.A.L. Geo Extraction

L.O.C.A.L. (llama3.2:3b) runs on every article and extracts the primary Canadian province and city. If the agent returns no confident result, a heuristic regex fallback scans for province names and city names in the title and first 500 characters of body text.

Coverage goal: ≥ 80% of articles have province_code populated within 24 hours of ingest.

AGENT MODEL CARDS

F.A.C.T. — Orchestrator

gemma3:12b · News & Media dept

"You are F.A.C.T. (Factual Analysis, Credibility & Transparency), orchestrator of the What The Fact AI newsroom. Score each article for priority analysis…"

B.I.A.S. — Bias Analyst

qwen2.5:7b · News & Media dept · upgraded 2026-05-28

"You are a Canadian political bias analyst. Score this article's political framing on the Canadian spectrum (LEFT=NDP/labour, CENTER-LEFT=Liberal/CBC, CENTER=neutral, CENTER-RIGHT=fiscal conservative, RIGHT=Conservative/Postmedia). Avoid defaulting to 0.0 — make a decision even if uncertain. Return ONLY valid JSON: {score, band, rationale, trigger_phrases, confidence}"

V.E.R.I.F.Y. — Fact-Checker

gemma2:9b · News & Media dept

"Extract FACTUAL ASSERTIONS only (statistics, historical events, attributed quotes, study results). SKIP opinions and descriptive reporting. If article contains no factual claims, return overall_verdict='descriptive_reporting'. Return ONLY valid JSON: {claims: [{text, verdict, reasoning, confidence}], overall_verdict, confidence}"

L.O.C.A.L. — Geographic Intelligence Analyst

llama3.2:3b · News & Media dept

"Extract the primary Canadian location from this news article. Return ONLY valid JSON: {province_code, city, lat, lng, geo_scope, geo_confidence}"

KNOWN LIMITATIONS

LLM analysis is based on training data (cutoff 2024) — models do not have internet access or real-time fact databases.
Bias scoring reflects the LLM's interpretation of language patterns; it may not match human expert consensus in all cases.
Geolocation uses source province as fallback when AI confidence is low (<0.5).
Not all articles receive full AI analysis — only priority articles (top 20/cycle) get LLM treatment; all others receive heuristic + source-editorial bias scoring.
Outlet bias profiles are rolling averages; new outlets are seeded from editorial stance data and refined over time.
Paywalled articles (Postmedia, Torstar) have limited body text — AI analysis relies on title + Open Graph description only.
DistilBERT Signal 3 (US-trained NLI model) may not map cleanly to Canadian political framing; treated as a second opinion, not ground truth.

COVERAGE GAPS — OUTLETS WE CANNOT MONITOR

Why this matters

WTF can only analyze outlets that publish open RSS feeds. Several major Canadian news operations have discontinued public RSS access, making their journalism invisible to independent algorithmic scrutiny. We document every gap — because transparency about what we cannot see is as important as what we can.

CTV News — Bell Media ✓ ACTIVE

Status: Restored and active as of 2026-05. CTV News RSS feeds are functional and ingesting successfully (~867 articles/week).

Coverage note: CTV News is Canada’s largest private broadcaster. Bell Media (parent) also operates BNN Bloomberg, TSN, and 35+ radio stations. CTV articles receive full B.I.A.S. + V.E.R.I.F.Y. analysis. Outlet editorial stance: center.

La Presse

Status: RSS discontinued. All feed URLs return 404 as of 2026-04-29.

Why it matters: La Presse is the dominant French-language daily in Quebec — the largest French-speaking market in North America. Without La Presse, WTF has a significant blind spot in Quebec journalism and French-Canadian political coverage.

Likely reason: La Presse went app-first in 2018 (La Presse+), an ad-supported tablet app. RSS access would bypass their app distribution entirely.

What we’re investigating: Radio-Canada RSS (CBC French) as a partial replacement for Quebec coverage.

Toronto Star

Status: RSS URLs return HTTP 200 but serve HTML, not XML. Feed is effectively non-functional as of 2026-04-29.

Why it matters: The Toronto Star is Canada’s highest-circulation print newspaper and a historically progressive voice. Its absence creates a gap in center-left Ontario coverage.

Likely reason: Torstar went behind a hard paywall in 2020. Open RSS would let readers access headlines for free, undermining subscriptions.

Active sources (as of 2026-05-28)

WTF currently ingests from 46 active Canadian sources across English and French, spanning major metro dailies, public broadcasters, regional outlets, Indigenous press, and independent investigative media.

Sources span the full political spectrum: left (Halifax Examiner, Ricochet, NB Media Co-op, PressProgress), center-left (CBC English/French, Radio-Canada, Le Devoir, Toronto Star), center (Globe and Mail, Global News, The Tyee, Canadaland), center-right (National Post, BNN Bloomberg, Financial Post, Postmedia regional), and right (National Post editorial). 18 beats monitored including Parliament, climate, crime, healthcare, and Indigenous affairs.

Seeking additional sources: Quebec French-language press (Le Soleil, Journal de Montréal), Atlantic regional, and Northern/Indigenous-owned media. Contact hello@what-the-fact.com if you represent a Canadian outlet with an open RSS feed.

FLAG A SCORE

Think we got it wrong?

Flag any article score you believe is inaccurate. All flags are reviewed by V.E.R.I.F.Y. and factored into model improvement.

Use the "Flag this score" button on any article page, or POST to /api/v1/flag

LIVE ARTICLE SCORER

Score any text

Submit any news text to the WTF pipeline and get a full bias + fact-check + geo analysis. Rate limit: 5 requests/day per IP.

Try the live scorer →