Forecast accuracy
Forecast accuracy scorecard
World Monitor logs every forecast it publishes and aims to score each one once its outcome is known. While the audit below is open, this page publishes no scores.
Under audit since 2026-10-07. An audit found three errors in how forecasts were scored. Some outcomes were recorded as "did not happen" without reading the data that decides them. Some forecasts were counted more than once. Some were scored at a probability other than the one published. The scores previously shown here were not reliable and are withdrawn while corrections are made. Forecasts are still being published and logged, and their outcomes will be rescored once the fixes land.
The findings and the fixes are tracked in issue #8990. The download below keeps the raw figures and marks them as under audit.
Methodology
This is the scoring method as documented. The audit found places where the scoring did not follow it.
Brier/log score over resolved YES/NO published forecast windows; VOID and pending entries are counted for coverage but excluded from accuracy math. The ledger totals and the maturity funnel count published forecasts only: shadow bets, scored for evidence but never shown, are left out of them. Each window is one question, and it is scored on the probability published when the window opened; a later re-emission of the same question does not change it. While an outcome-fitted calibration gate passes, that probability is calibrated, and the API does not mark which ones are; after a switch between raw and calibrated publication, the rolling window mixes forecasts published under both. 120 forecasts scored against a data feed we could not read correctly are voided and left out of every score (issue #5233). 322 shadow bet windows that opened on a base-rate placeholder instead of a model forecast are left out of every count, VOID included (issue #8990). Since 8 October 2026, 11 of 17 resolved published forecasts were void (64.7%, 95% interval 41.3% to 82.7%). That is too few to compare with the target of under 15%, which needs at least 30 resolved. Judged forecasts resolve days after hard ones and have voided more often, so early readings run low. This counts published forecasts first issued on or after that date, so it leaves out unpublished shadow bets and the voids relabelled in the issue #8990 audit.
Recently resolved forecasts, unverified
| Forecast | Made | Chance scored | Outcome | Resolved | How it was settled |
|---|---|---|---|---|---|
| GPS interference in Red Sea shipping zone | 2026-10-15 | 58% | Did not happen | 2026-10-24 | GPS jamming reports read 1 |
| Grain/Energy volatility from Russia instability | 2026-09-23 | 22.4% | Did not happen | 2026-10-24 | Commodity prices read 627.5 |
| Within the 7d horizon, did Syria experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Escalation risk: Syria"? | 2026-10-15 | 62.8% | Void | 2026-10-24 | The judges disagreed |
| Will "Military posture escalation: Black Sea" (military, Black Sea) resolve YES within its 7d horizon? | 2026-10-14 | 41% | Void | 2026-10-23 | The judges disagreed |
| Within the 30d horizon, did Nigeria experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Nigeria"? | 2026-09-20 | 93% | Happened | 2026-10-23 | Judged against archived news: Nigerian troops kill 103 suspected terrorists, destroy logistics hubs – Official |
| Within the 30d horizon, did Sudan experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Sudan"? | 2026-09-21 | 65.2% | Happened | 2026-10-23 | Judged against archived news: Drone strike kills more than 150 at market in Sudanese war |
| Within the 30d horizon, did Yemen (North Yemen) experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Yemen (North Yemen)"? | 2026-09-22 | 85% | Happened | 2026-10-23 | Judged against archived news: Houthi ballistic missile strike kills at least 53 Yemen govt troops |
| Within the 7d horizon, did Northern Europe experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Theater escalation: Northern Europe"? | 2026-10-13 | 40% | Void | 2026-10-21 | Both judges found the evidence insufficient |
| Supply chain disruption: Strait of Hormuz | 2026-10-13 | 52.5% | Happened | 2026-10-21 | Shipping chokepoint status read 85 |
| Will "Military posture escalation: Northern Europe" (military, Northern Europe) resolve YES within its 7d horizon? | 2026-10-12 | 41% | Void | 2026-10-21 | Both judges found the evidence insufficient |
| Will the Bank of Israel make no change to the Bank of Israel Interest Rate after the October decisio | 2026-09-25 | 80% | Void | 2026-10-21 | The feed showed the market price, not how the market resolved |
| Within the 30d horizon, did Ukraine experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Ukraine"? | 2026-09-20 | 85% | Happened | 2026-10-21 | Judged against archived news: Russian strike kills at least 33 in eastern Ukraine in deadliest single attack this year - CNN |
| Will "Military posture escalation: Eastern Mediterranean" (military, Eastern Mediterranean) resolve YES within its 7d horizon? | 2026-10-11 | 41% | Void | 2026-10-20 | The judges disagreed |
| Will "Military posture escalation: Israel/Gaza" (military, Israel/Gaza) resolve YES within its 7d horizon? | 2026-10-11 | 41% | Void | 2026-10-20 | The judges disagreed |
| Will "Unknown-linked airlift surge near Iran Theater" (military, Middle East) resolve YES within its 7d horizon? | 2026-10-12 | 34% | Void | 2026-10-20 | Both judges found the evidence insufficient |
| Within the 30d horizon, did Mali experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Mali"? | 2026-09-18 | 93% | Void | 2026-10-20 | The judges disagreed |
| Within the 7d horizon, did China experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Escalation risk: China"? | 2026-10-12 | 22.8% | Void | 2026-10-20 | Both judges found the evidence insufficient |
| Within the 7d horizon, did Russia experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Escalation risk: Russia"? | 2026-10-11 | 58.4% | Void | 2026-10-20 | The judges disagreed |
| Within the 7d horizon, did Ukraine experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Escalation risk: Ukraine"? | 2026-10-12 | 75.2% | Happened | 2026-10-20 | Judged against archived news: Russian strikes kill civilians, injure dozens across Ukraine |
| Within the 30d horizon, did Pakistan experience a materially escalated level of armed conflict versus its recent baseline, consistent with "Active armed conflict: Pakistan"? | 2026-09-18 | 85% | Void | 2026-10-19 | Both judges found the evidence insufficient |
Market alerts: did the news follow?
The audit did not cover this section. Market alerts are scored from a separate ledger, and these figures have not been rechecked.
World Monitor raises a market alert when a market or a prediction market makes an unusual move. Some alerts fire when there is no news behind the move, and one type fires when related news is already out. Each alert is checked 6 hours later. It counts as a hit if an established news outlet published a new story about the same company, commodity or topic in that time. The same check also runs on the same market for a stretch of the same length one day earlier, when no alert was raised, and that gives the base rate. An alert type is useful only when its hit rate is clearly above the base rate on the same alerts. The figures cover the last 30 days and were generated 2026-10-26 05:15:25 UTC.
| Alert type | Alerts scored | News followed | News followed, compared alerts | News a day earlier, same markets | Typical wait for the news (median) |
|---|---|---|---|---|---|
| Prediction market moved on a quiet news day | 1 | Not yet measurable | Not yet measurable | Not yet measurable | Not yet measurable |
| Market moved alongside related news | 214 | 27.6% of 214 alerts | 25.0% of 96 alerts | 28.1% of 96 earlier windows | 1 h 11 min |
| Market moved with no news found | 590 | 1.7% of 590 alerts | 3.7% of 214 alerts | 2.3% of 214 earlier windows | Not yet measurable |
| Energy price rose with no pipeline news | 20 | Not yet measurable | Not yet measurable | Not yet measurable | Not yet measurable |
How an alert is scored
An emission resolves HIT when a story first tracked by the English news digest within six hours after the emission has at least one Tier 1 or Tier 2 source and its title names the same entity: for a market, an alias match for the symbol's registry entity or a whole-word match of the market name of four or more characters; for a prediction market, a topic keyword that appears in the question itself. It resolves MISS when six hours pass with no such story. It resolves VOID, and is not counted, when the story archive for its window expired before the resolver read it. Lead time is the gap between emission and the matching story's first tracking. The base rate applies the same rule to a control window of the same length for the same entity that starts 24 hours before each scored emission, counted only when the archive covers the control window, no alert for that entity was open during it, and the emission and the control fall on the same side of a weekend in UTC.
Open the live forecast panel in World Monitor →Related reference
Download: scorecard.json. Source: World Monitor forecast scorecard snapshot. The raw figures in the download were generated 2026-10-25 11:11:45 UTC and read on 2026-10-26, and are under audit. Live results come from the credentialed forecast scorecard endpoint, which this page freezes so it can be read without one.