How the Analyst Analyzer works

Fair to the people being rated, and useful to you: who is actually ahead of the news?

Win totals

A stated range counts as said (“between 48 and 52”, “high 20s to low 30s”). A single number (“about 51 wins”, “they go 52-30”) counts as right within two wins either way, so 49 to 53 for “about 51”. Win totals settle once all 82 games are played.

What gets listed

Only calls we can verify. A call is listed when it is firm and checkable (a player is out, starts, plays big minutes; a team wins a game or a number of games; a trade happens by a date), when we hold the data to settle it, when it passed our automatic accuracy check against our notes, and when it traces to the source: a post with its link, or a podcast moment whose words matched the transcript. Opinions, "could" and "might" calls, and calls we can't check yet (awards, seeds, contract extensions) are counted on each profile but not listed.

How we write it down

We turn each call into a plain statement with a subject, a deadline and a published rule, in our own words. A call that needs a number the speaker never gave is not scored unless one of our published rules fits it.

Hedged calls

"I think" and "probably" count. "Could", "might" and "wouldn't be shocked" are counted as floated and never scored either way. Each voice shows a commitment rate: how many of their calls were firm and checkable.

Expected chance

Every settled call gets the chance it had of happening based only on what was public when it was said: the official injury report, the betting line, or the player's recent games. Saying a player who is already listed Out will sit earns almost nothing; calling a surprise earns a lot.

Score

For each call, credit = result (1 or 0) minus the expected chance. Calls about the same game and player count as one. The average credit is pulled toward zero when there are few calls, and comes with a likely range.

Grades

A +8 or more · B +3 to +8 · C −3 to +3 · D −8 to −3 · F below −8 (percentage points above what was expected). A voice is graded once they have 25 independent settled calls across at least 2 kinds, and at least 25% of their calls were firm. When the likely range crosses a grade line we show both.

Trust score (experimental, trust-v0)

A 0–100 number for how far you can rely on a person's listed record. It blends prediction skill (half the weight: 50 plus 300 times their shrunk skill, kept between 0 and 100), how often their first news reports proved right (30%), and how often they commit to firm, checkable calls (20%, full marks at 50% of their calls, so piling up obvious calls stops helping). It shows once 10 independent calls have settled, with a likely range, and is marked early until the voice is graded. Volume and recency never raise it. Shows, surveys and record-only voices get none. Flags never lower it: a flagged call simply leaves the score until reviewed, and mistakes that were ours (like crediting the wrong person) are fixed, not charged to anyone.

Kept apart

Predictions and news reports are rated separately. Reports only count when they were first: repeating news that was already public is shown but not scored. Preseason games are shown but not scored.

Who said it

We credit a person only when we are sure: their own account, or a podcast speaker identified by name. When we cannot tell which host spoke, the call goes to the show and never to a person.

Flags and corrections

Every listed call has a Flag link. "The result is wrong", "it's listed twice" and "it isn't a prediction" are rechecked automatically; the rest are reviewed by a person within 14 days. While a call is flagged it shows "Under review" and is left out of every score. Results are never edited in place: a correction is a new version with a reason, and every upheld correction is in the public corrections log. A voice can ask for a correction, a record-only profile or an account change through "Is this you?"; we verify the request before anything changes.

Audit

Every listed call gets a second, independent check against the recording; 583 of 629 checked so far passed (the rest are under review or were corrected). We also audit a random 5% sample (at least 30 calls) of each version of our extraction; above 5% errors, that version's grades only count individually checked calls.

props-v1: 3 of 30 sampled calls needed correction (10%), 2026-10-19 · grades from this version count only calls that passed the individual second check