Top after five games. Are they really the best? Where priors come from
Early-season league tables mislead. Over 550 Scottish team-seasons, only a third of a team's early form carried on. Bayesian thinking predicted exactly how much, using a prior measured from the data.
Intermediate Part 2 of Bayesian Football
Contents
The football question
Five games in, your team has 13 points from a possible 15. Top of the league. The pundits are asking whether they can keep it up.
How good are they really? And how would you even begin to answer without waiting for the rest of the season?
The concept
Part 1 combined a prior (what you believed before) with evidence (what you've just seen). For a penalty taker, the prior was a sensible guess: most takers score about three in four.
For a league table there's a better way. We don't have to guess the prior; we can measure it from past seasons. How much do teams really differ in quality? How much do results bounce around from game to game? Those two numbers tell us how much to trust five games. Using history to set the prior is called empirical Bayes.
What the data says
Take every team in every season of the Scottish Premiership and Championship since 2000/01: 550 team-seasons. For each one, compare its points per game over the first five matches with its points per game over the rest of the season.
On average, only about a third of a team's early gap from the league average carried on. The other two-thirds was noise.
| Games played | Share of the early gap that carried on | Bayesian prediction |
|---|---|---|
| 5 | 33% | 30% |
| 10 | 48% | 47% |
The "Bayesian prediction" column comes from a formula that needs only two numbers: how much single results vary, and how much team quality varies across the league. It never compares any team's start with its finish, yet it lands within three points of what actually happened.
Were the early leaders the best?
Sometimes. Across 50 league-seasons, 60 teams were top (or joint top) after five games. Only 42% of them finished top. The typical early leader finished second, and five finished in the bottom half.
A football example
Our team has 13 points from 5 games: 2.6 a game. The league average is about 1.37 points a game.
The data says to keep a third of the gap:
$$\begin{aligned} &1.37 + 0.33 \times (2.6 - 1.37) \\ &\approx 1.78 \text{ points a game} \end{aligned}$$
Good, clearly better than average, but nowhere near the 2.6 they started with.
In plain football
- 1.37 is what an average team takes per game in these leagues.
- 2.6 − 1.37 is how far above average their start was: 1.23 points a game.
- 0.33 says only a third of that is likely to be real. The rest is the luck of five fixtures.
- So expect about 1.78 a game from here: a top-half side, not a champion in waiting.
The same weighted average as Part 1
In Part 1 the best estimate was a weighted average of the prior and the record, each weighted by how much evidence it was worth. The same thing works here. Measured from the data, the league average is worth about 11 games of evidence:
$$\begin{aligned} &\text{best estimate} \\ &= \frac{5 \times 2.6 + 11.4 \times 1.37}{5 + 11.4} \approx 1.75 \end{aligned}$$
That's within a whisker of the 1.78 from the data, from a completely different route. After five games the league average has more than twice the say of the team's own results. By about 11 games the two are equal, and from then on the team's own record starts to win.
Show the mathsWhere the 11 games comes from. Optional.
A team's points per game over n matches is its true quality plus noise. The noise shrinks with more games:
$$\text{Var}(\text{observed}) = \sigma^2_{\text{true}} + \frac{\sigma^2_{\text{game}}}{n}$$
In this data, single results vary with \(\sigma^2_{\text{game}} \approx 1.73\) (points are 0, 1 or 3, so they jump about), while real quality varies across teams with \(\sigma^2_{\text{true}} \approx 0.15\), a standard deviation of about 0.39 points a game.
The Bayesian estimate keeps a share \(k\) of the observed gap from the average, where
$$\begin{aligned} k &= \frac{\sigma^2_{\text{true}}}{\sigma^2_{\text{true}} + \sigma^2_{\text{game}}/n} \\ &= \frac{n}{n + \sigma^2_{\text{game}}/\sigma^2_{\text{true}}} \end{aligned}$$
So the prior is worth \(\sigma^2_{\text{game}}/\sigma^2_{\text{true}} \approx 1.73 / 0.152 \approx 11.4\) games. With \(n = 5\): \(k = 5/16.4 \approx 0.30\). With \(n = 10\): \(k \approx 0.47\).
Why it matters
This is regression to the mean, explained. Teams that start unusually well tend to come back towards the pack, not because they "bottle it", but because part of every good start is luck, and luck doesn't carry on. The same logic applies to the bottom of the table: a side with one point from five games is rarely as bad as that.
It's also why good analysts don't read early league tables at face value, and why a manager sacked after a poor first month has often been judged on noise. Empirical Bayes puts a number on how much noise there is.
Limitations
- Teams change during a season. Injuries, signings and new managers move true quality, so later results can differ for real reasons, not just noise.
- "The league average" is a blunt prior. We know more than that before a ball is kicked: last season's finish, the budget, the squad. A prior that uses them would be sharper. That's the idea behind hierarchical models, a later part of this series.
- Two Scottish divisions. Leagues where quality is more spread out would give the early table more weight; tighter leagues, less.
Try it yourself
Take the league table after five games this season. For each team, work out points per game, then keep a third of the gap from about 1.4 and add it back. That's a far better guide to the final table than the one on the screen. For the penalty version of the same idea, try How good is he really? with "How sure" set to 11.
Further reading
- Empirical Bayes method, Wikipedia. Setting the prior from the data itself, as done here.
- Regression toward the mean, Wikipedia. Why extreme starts drift back towards average.
- Stein's example, Wikipedia. The surprising result that shrinking many estimates towards their average beats taking each one at face value.
- Seeing Theory: Bayesian inference, Brown University. Interactive: watch a prior and data combine.