Once the season is over and done with I’ll be looking back at how the EI has performed and how well it’s predictions compare with the bookmakers so look out for that next week!

I think we are all aware that the data provided to the BBC by the Press Association isn’t perfect. I certainly have more major misgivings about using it for smaller sample sizes, and particularly the numbers from lower leagues, than I would with data from other providers. It does, however, have two things going for it – it’s cheap enough for a grad student to afford (i.e. it’s free), and there’s tons of it. now given enough data I think that a lot of the quirks will work themselves out* – and finding strong season-to-season correlations within the data suggests this is the case, so until something better comes along I’m happy to use it once a sample is large enough.

One thing I find particularly irksome is that penalties don’t count towards as shots, and in theory this is going to alter a teams metrics. Whilst it’s only going to alter a teams TSR by a maximum of 0.5% (eg changes from 0.607 to 0.606, or 0.442 to 0.444 were about the most egregious examples I could find in a given season), there’s more scope for the luck driven metrics, such as sh%, sv%, and PDO (this is the seminal piece of that series), to be shifted.

Fortunately the nice people at Infostrada (whose website may be found here) provided me with a ton of penalty data last year that I used for a series of posts (a summary of which may be found here), and I can use that data to strip successful penalties from the PA data. (There’s no need to strip missed/saved penalties as they aren’t recorded in either the goal or shot columns).

Why does this matter? Well penalties are a different animal to your average shot. According to the PA data, the average shot on target in the Premiership has a ~20% chance of being scored, whereas penalties are scored, pretty uniformly, 78% of the time. So if a team has an unusually high proportion of shots being taken as penalties then they’ll be getting a boost in their sh%. Obviously the opposite will occur to a teams sv% if they a larger number of shots against them are penalties.

I had a good stab at determining whether the ratio of penalties a team received was related to TSR in the first plot here, and found that there was a strong general trend (albeit with a couple of notable exceptions). This time I’m going to go about it in a different way and see how the answers compare.

I’ve taken each of the 38 teams that were in the Premiership between 2001-02 and 2011-12, and ordered them in terms of their TSR over that span. (The sample consists of ~1,000 penalties, ~11,000 goals, ~100,000 shots on target, and ~200,000 shots in total). I’ve then gone down the list and created four bins of teams – with each bin containing as close to 50,000 shots as possible. As such bin 1 contains the best teams (those of highest TSR), and bin 4 contains the worst (those of lowest TSR). A table of the shots and penalties for each bin is below. To make the writing large enough to be legible I’ve substituted ‘+’ in for ‘for’, and ‘-’ in for ‘against’ **. Click on the table for a larger version.

To me that table reads pretty interestingly. The best teams take 61% of the shots in their matches, and are awarded 61% of the penalties. That makes perfect sense to me, and means that (outside of possibly United – see the plot in the prior link) you’re going to have to make a hell of a case if you want to argue that big teams get a greater proportion of their penalty appeals awarded than smaller teams.

Bin 3 also lines up nicely (47% in each category), however this isn’t the case for bins 2 and 4. Looking at it you could even suggest, given that the proportion of penalties won by the teams in bin 2 is low by roughly the same amount as bin 4 is high, that the ‘awful’ teams (bin 4) are being awarded penalties at the expense of ‘good’ teams (bin 2).

And that was my first thought too. Plus, a 2-3% difference between TSR and % of pens + seems like it could add up to a significant amount. But is that the case, and how does this translate into points in the league? Lets calculate how many of the penalties we’d expect the team in each bin to earn, based solely on their TSR:

Given that each bin is comprised of ~55 team seasons we can conclude that, as a group, the teams finishing between 6th and 10th in the table (represented by bin 2) are sold short to the tune of ~2 penalties per season. ~2 penalties X 78% conversion rate X ~0.6 points per goal = ~1 point. Split that five ways between the teams that finish 6th-10th and it comes to ~0.2 points per team. So whilst it may well be a real effect it’s not something they should be losing sleep over, especially given that most (>66%) of the benefit is felt by teams who they aren’t in direct competition with (those at the bottom of the table).

I’d like to look into this effect further (i.e. whether a team in bin 4 is more likely to get a penalty in a game against a team in bin 2, and vice versa) however it’d take a ton of work, and if we slice a sample of 1,000 penalties 16 ways we’re looking at a sample of ~60 penalties. A sample that size is unlikely to yield a statistically significant result.

So what can we conclude? Well, whilst there are outliers in terms of individual teams, penalties are pretty well correlated to TSR. As such we’d expect to see pretty similar drops in sh%/climbs in sv% for each team when the penalties are stripped out of the PA data. That’s something I’ll take a look at next time.

*There are innumerable ways to test this but I simply don’t have the time

**The eagle eyed amongst you will have spotted that the ‘shots’ columns don’t add up. I’ll go through and find the discrepancy at some point but it’s <200 shots out of 200,000 – I doubt that <0.1% of the data would shift the results much in this case.

While Ferguson’s achievements are significant, they do help fuel one of the consistent criticisms of European soccer leagues – their predictability at the top.  The fact that teams can spend as much as they like and compete within a balanced 34 to 38 game schedule means that over time the teams who spend the most end up being the only ones who have a shot at winning the league title.  What else would explain the level of investment made by benefactor owners at the likes of Chelsea, Manchester City, and Paris Saint-Germain?  The reality is that in the Premier League’s entire history there have only been two champions outside the top three in the Transfer Price Index’s Total Team Valuation (annual salary bill + player transfer valuation), and they were the 1995/96 and 1996/97 Manchester United teams fueled by that side’s golden youth generation.  This season was like many others before it, with the predictability of such money-fueled outcomes leading ESPN to call all four of the major European leagues with one third of the season left to play in February.  While not the primary reason for UEFA’s Financial Fair Play rules, it has been shown that the side effect of such rules will be to constrain the competitive benefit big spenders can derive for themselves and as a consequence make each of the top leagues in Europe a bit more competitive.

Legality of Financial Fair Play aside, UEFA does not need to actually go to such extremes to level out the playing field when it comes to who wins league titles.  Authors Stefan Szymanski and Simon Kuper dispelled the myth that the outcome of soccer leagues like the EPL are more predictable than other sports leagues in Chapter 8 of their first edition of Soccernomics.  In that chapter, entitled Football vs. Football, Szymanski and Kuper showed that the NFL had dominant teams who won 70% or more of their regular season games just like Manchester United, Chelsea, and Arsenal did over the first decade of the 2000’s.  They also showed there are NFL teams just as futile as Bradford City was in the Premier League.  Using statistical tests, they demonstrated that both leagues are similar in how they distribute wins throughout a season.  The key difference between the two was that the NFL randomized its outcomes a bit better by running a shorter season (in statistical speak that is known as “restricting sample size”) and by using a playoff system (allowing the randomness of single events to dominate).

The fact that most soccer leagues use a balanced home-and-away schedule with a larger number of matches than American football makes them a statistician’s dream.  Such a seasonal format also seems to strike the proper balance between allowing too much dominance of random events (a 16 game NFL season) and the pointlessness of a much larger number of matches (try American baseball’s 162 game season followed by playoffs).  Rather than shorten or lengthen the season of Europe’s leagues, why not look to a playoff system to help randomize league championship outcomes?  What effect might that have on the concentration of titles versus the current format where only five teams have won the twenty-one contested Premier League trophies?

There are many ways to model such a possibility.  This blog utilized the Transfer Price Index’s m£XI model that looks at venue and relative squad costs to come up with projected win/tie/loss likelihoods for a match.  Data from the last twenty seasons was used to construct a 10,000 run Monte Carlo simulation to estimate what effect a playoff in the EPL might have had on the title’s history.  Here were the rules that were used to set up the simulation:

1. Only the top four teams were used in each season’s playoff.  This was done very purposefully given leagues in North America err by letting in too many teams into the playoffs (up to half the league in some instances!).  The purpose of this playoff simulation is to observe what might happen if the top four teams who realistically had a shot at winning the title throughout the season were given a final shot at winning it via a playoff.
2. Each playoff round is a single elimination match.  This was done to add a bit of random behavior to the outcome.
3. Regular season point totals determine seeds within the playoffs, and the team with the greater number of points hosts the playoff match.  This applies to both the semifinals (first round) and final (second round).  Such a home pitch advantage is the reward for sustained success throughout the season.  Finish top of the table, and the club plays at home for the entire playoffs.  Finish fourth, and the club must win two away matches in a row to win the title.
4. Seeding in the semifinal round is 1 versus 4 and 2 versus 3.  The highest seed to make it out of the semifinal round hosts the final.
5. A correction had to be made to account for injured players as well as ones being rested during meaningless final matches as historical £XI data was being used within the simulation.  This was done by using the average £XI values for the final ten matches of each of the teams in the playoffs.

The results from 10,000 simulations demonstrate just how much a four-team playoff would have changed the title count of the last twenty years of Premier League play.  The table below summarizes the average number of titles won by each club within the simulation.

The most impacted club in such a playoff scenario is Manchester United.  Don’t expect any Red Devils supporters to get behind the idea of playoff anytime soon.  They would most likely go from the twelve titles they had through last season down to seven, and would have a 2% chance of holding on to the twelve they have won.  Some might argue that Alex Ferguson might have still won more than seven titles even with a playoff, but his track record in other knockout competitions, especially the Champions League, suggests he too would succumb to the randomness that is a small sample size knockout tournament against top competition.

Interestingly, Chelsea and Arsenal finish in the top four enough that they would likely maintain the three titles they had each won through last season.

The first club to greatly benefit from such a playoff format is Liverpool.  Having never won a Premier League title, Kopites might get behind a playoff system if they knew they would have most likely won two titles via such a playoff with a less than 10% chance of having no titles after twenty years.  Newcastle United, who were one of the top five spenders in the league between 1996 and 2007, would have likely won a title thanks to the several top four finishes that came about from such spending.  Manchester City and Blackburn Rovers would retain their sole titles, while Leeds United and Aston Villa rounded out the final few teams most likely to win a single title.  Who knows if a title would have been able to stave off Leeds’ poor financial management that sent them tumbling through the Football League ranks or stop the slow decline Aston Villa has experienced over the last half decade.  For the time being, however, it would have added a large sense of pride to two clubs who came close to winning a title in the league’s first decade.

A playoff would not only help expand the number of clubs with Premier League titles to a total of nine, but it would also inject some more excitement into the end of the Premier League season.  Imagine what a four team playoff would be able to draw in terms of TV revenue, fan excitement, and media coverage over two extra weeks of play!  The EPL could distribute most of the derived revenue not to the teams competing in the final four matches, but to the other sixteen competing in the Premier League.  Such a solidarity payment would help close the gap between the big spenders and what seems to now be “also ran’s.”  As if staging a playoff to decide the Premier League title weren’t exciting enough, the league could also use the results to determine who gets the automatic byes into the Champions League group play stage versus who must earn their spot via the play-in round.  This could be accomplished by having the two losers of the semifinal round of the playoffs face off in a third place match, with the stakes of the match being even higher in years in which the Premier League was only afforded three Champions League slots.  The playoff could even be made to favor a bit more random of an outcome by utilizing the top six teams where the top two teams get a bye into the semifinals while teams three through six face each other in a quarterfinal round.

Thus, a playoff system could accomplish two of UEFA’s goals.  First, it would create a bit more random outcome in which a greater number of clubs would win league titles.  Secondly, such a playoff could be used to shore up the financial solvency of teams looking to spend a bit beyond their current means to earn a fourth place position and a lucky chance at winning a Premier League title.  North American salary caps won’t work given the worldwide nature of the game and the relatively free movement of labor.  Financial Fair Play may run afoul of EU law, and even if it doesn’t it may largely serve to calcify the current structure of “haves” and “have-nots”.  If UEFA and any of its constituent national federations and leagues want to find a less intrusive way to inject less predictability into league outcomes, perhaps they should start with a playoff instead.

Arsenal has an 85% chance of finishing in the Top Four, while Spurs have a 15% chance.  Bookmakers are taking a bit more conservative approach.  If you do the math on (Probability of Spurs Win x Probability of Arsenal Tie) + (Probability of Spurs Win x Probability of Arsenal Loss) you will find bookmakers see Spurs having a 29% chance of finishing in the Top 4 and Arsenal a 71% chance, or a nearly 2.5:1 advantage in odds for Arsenal.  Either way you look at it, Arsenal is in the driver’s seat for Champions League play.  Of course one could argue that anything that happens this coming weekend could be chalked up to luck with such a small difference between the two sides’ point totals , but either way it will be exciting to watch!