Things We Think We Know About Football – July 2013

Some days it’s good to take a step back from the daily deluge of information and try to organize what you think you have learned recently. I’ve been writing regularly on football analytics for about six weeks now (and off and on for seven months), and I’d say half the topics I’ve covered are based […] Read more ›

What the Hell Are We Doing? Goals and Transfer Shopping

Today’s piece is inspired by the early summer transfer signings. The goals in the title don’t refer to actual goals that are scored on the pitch, though that’s what I normally write about. Today, the word “goals” is referring to team milestones. Literally, what is your team trying to accomplish this year? Are they simply […] Read more ›

The Politics of Analytics: Raptors, Dinosaurs, and Matt Busby

Whether football likes it or not, analytics has been creeping into every aspect of a club’s operations – from scouting to medical to match day analysis. And yet, it is fair to say that analytic approaches have not been welcomed with open arms. Aside from scouts who have greeted the analytics movement with cold-shouldered trepidation, some of the greatest resistance to analytics has come from members of the coaching profession. So it was refreshing to hear Bill Gerrard, one of the leading practitioners of the dark arts of analytics in football (rugby and association), talk about his work with the coaching staff at Saracens F.C., one of the leading teams in English rugby union.
Speaking at the Sports Analytics Innovation Summit in London, Gerrard explained how he, along with the Saracens staff, have developed a system he terms “evidence-based coaching” to assist in the team’s evaluation of the coaches’ game plan, and to use data collected by and with the help of coaches on how well the players have performed relative to the game plan. This information, after careful vetting and analysis, can then also be fed back to players to see how they are performing relative to the coaches’ expectations.
Aside from the obvious and perhaps not so obvious disadvantages of the system (I’ve described some of them here), the tale Gerrard was telling was both refreshing and predictable. Refreshing, because Gerrard explained how well the team of coaches and analysts were working together using both qualitative and quantitative data, and in the spirit of improving the team. Predictable, because his tale highlighted some of the key obstacles to analytics in modern-day football clubs.
Don’t get me wrong; evidence-based coaching is a good idea. Using more information is better than using less; having quantitative and qualitative measures to gauge performance (along with our old friends gut and instinct) helps coaches and players figure out what really happened; and so on. But evidence-based coaching limits analytics because it leaves coaches in charge.

When it comes to analytics, coaches are some of the toughest nuts to crack.

The politics of analytics get in the way of turning Gerrard and others like him, into more than mere data assistants. By inclination and training, coaches are much more focused on what they can see with their own eyes – games, videos, etc. – and prefer to rely on their own experiences and instinct, rather than what hard data can tell them. They are like the ultimate old-line scouts in Moneyball. Moreover, they like to have and exercise control.
Mind you, football or rugby are far from unusual. In fact, they’re a lot like baseball or basketball.

Take the case of the Toronto Raptors of the NBA. Zach Lowe, a writer for ESPN’s Grantland recently wrote a revealing story about the Raptors’ amazing and creative use of new (big) data and the race to get the most out of a new source of information for teams. Much of the story was – ostensibly – about the behind the scenes work the Raptors staff had done to unearth insight from “a camera-tracking system, known as SportVU, that records every movement on the floor and spits it back at its front-office keepers as a byzantine series of geometric coordinates. Fifteen NBA teams have purchased the cameras, which cost about $100,000 per year, from STATS LLC; turning those X-Y coordinates into useful data is the main challenge those teams face.”

But dig a little deeper, and organizational politics rears its head, with coaches in the thick of it. The Raptors have approached the new data by getting the coaches and analysts to work together. To understand each play or game situation, they have devised a “ghost system” – a system of ghost players who “are doing what Toronto’s coaching staff and analytics team believe the players should have done on this play – and on every other Toronto play the cameras have recorded. The system has factored in Toronto’s actual scheme and the expected point value of every possession as play evolves.”
In principle, this is the basketball equivalent of Gerrard’s evidence based coaching – albeit with hi tech bells and whistles.
As the Grantland story makes clear, the Raptors have worked hard to develop a system that coaches can work with, investing significant resources in visualizing the data for coaches and players alike. But at its core, it sounds like a version of evidence-based coaching. As Lowe notes,
It is a very impressive piece of work. ‘Most teams are using spreadsheets or just using our reports,’ says Brian Kopp, executive vice-president at STATS. ‘The Raptors go a step beyond that, which only a few teams are doing, and their visualizations are the best I’ve seen.’

But even in a league and sport that have embraced analytics and in a club that’s made a serious investment in it, there are limits of an analytics-driven approach that have nothing to do with technical tools and everything to do with the politics of the organization.
The ghost system and what Lowe calls “some of the larger analytics-related issues” have caused friction between the front office and some of the coaches, with the coaches often skeptical of the value of the system and the approach. As Lowe explains,
The coaches, even the most receptive ones, seem to view analytics and SportVU mostly as a tool to confirm what they already think and know. Some samples: 

Dwane Casey, Toronto’s head coach: “It’s a good backup for what your eyes see.” Casey added, “It may also shed light on something else,” a sentiment both Nori and Sterner echoed at points. “But you can’t make all your decisions based on it, and it can’t measure heart, and chemistry, and personality.” 

Sterner: “It helps reinforce your gut. Most of the time, your gut is pretty much right.” 

Nori: “More than anything, it’s a tool to help confirm what your eyes see.”

In this way, basketball coaches and football managers are very much alike. As we explain in our forthcoming book The Numbers Game, there are a number of reasons why coaches desire to see the data as confirming what they already know – it’s actually what we should expect rather than something to be surprised by.

One reason for the resistance is psychological – it has to do with what psychologists call motivated reasoning. Coaches see what they want to see and they want to see what they already know. The problem is that the coaches’ visions can on occasion be inaccurate. As Keith Boyarsky [the team’s technical director of analysis] explains to Lowe
A lot of coaches will say how great it is that analytics confirm what they already see… The fact of the matter is, that’s not really true.

Rucker, the Raptors’ director of analytics provides an example of faulty thinking:
‘When you ask coaches what’s better between a 28 percent 3-point shot and a 42 percent midrange shot, they’ll say the 42 percent shot,’ Rucker says. ‘And that’s objectively false. It’s wrong. If LeBron James just jacked a 3 on every single possession, that’d be an exceptionally good offense. That’s a conversation we’ve had with our coaching staff, and let’s just say they don’t support that approach.’

The second reason for coaches’ resistance is political. Coaches like to have control. Football, in particular, has given managers considerable autonomy and control over football operations, and has done so for many years – managers are the true CEO’s of modern-day football clubs, and they sit on top of a pyramid they like to direct. 
In fact, there is a veritable cult of the manager that was first forged in England. A simple glimpse at the vocabulary of the world game offers proof enough of this: across football’s heartlands in South America, Italy and Spain, managers are still referred to as ‘Mister’; England sent out missionaries to Eastern Europe (Jimmy Hogan) and Scandinavia (George Raynor).

As Barney Ronay observes in his book The Manager: The Absurd Ascent of the Most Important Man in Football, the manager’s ‘shadow looms, fully evolved, now erect and miraculously walking on his hind legs: priest, messiah, hard-nut, patriarch and visible emblem of over a hundred years of confused and piecemeal progress’. As Ronay notes mischievously, the manager was England’s ‘gift to the wider world beyond its boundaries’.
The idolization was no less intense in the motherland. Arthur Hopcraft, in his newspaper columns and his 1968 book The Football Man, captured the rise of the manager perfectly. ‘To watch Sir Matt Busby move about Manchester is to observe a public veneration,’ Hopcraft wrote. ‘He is not merely popular; not merely respected for his flair as a manager. The affection becomes rapidly more deferential as they get nearer the man.’

When such is the manager’s status in the industry and within football clubs, it is hard to see why any manager would voluntarily surrender power. Ceding the mantle of expert knowledge means ceding power (and resources), and that’s something managers typically contemplate only under duress. Besides, analytics is the craft practiced by outsiders, pinheads, numbers guys, stattos, and geeks – not “real” football men.
In the end, evidence-based coaching allows coaches to adapt to a changing world that’s hard to ignore. From the standpoint of analytics, it is clearly better than nothing, but not as good as it could be; from the standpoint of coaches, it is as good as it’s going to get. Putting coaches at the center of the analysis enterprise means that there is always the risk that their motivations, incentives, and capabilities will overwhelm the team’s best interests.
Of course, this doesn’t therefore mean that – once freed from the shackles of coaches’ thinking and urge to maintain control – analysis and analytics will be a kind of deus ex machina that produces more wins by letting the nerds play Football Manager with real players and teams. But it does mean that taking analysis out of the hands – or more precisely, minds – of the coaches and managers or at least sharing the responsibilities of analysis opens up the potential for the hard and smart work and, importantly, the ideas of analysts to have a real impact on the club’s fortunes.
Two heads are better than one. And who knows?! Coaches and managers might breathe a sigh of relief when they are told they no longer have to know it all to begin with.

Read more ›

Statistical analysts and professional sports teams: a view from Bill Gerrard

I start off 2012 with a look at an article from 2011 — this piece in the Royal Statistical Society’s Getstats campaign by Leeds University professor Bill Gerrard on his experiences as a statistical analyst with professional baseball, soccer, and rugby teams. Gerrard had famously partnered with Billy Beane on some analytics work with the Oakland A’s ownership group but has since transitioned to rugby and cricket. It’s worth reading in order to appreciate better his role in the team and the challenges involved in integrating statistical performance analysis in a professional sports team. Some of the lessons learned are… Read more ›

My thoughts on Moneyball the movie

This afternoon I watched Moneyball at a theater in midtown Atlanta. I was looking forward to watching the movie not just because I had read the book, but also because I lived in the Bay Area during 2001-02 when the Oakland A’s were one of the best teams in Major League Baseball. So I remember the criticism that Beane weathered in the early part of the season, the growing excitement as the team embarked on their American League-record winning streak, and the crushing disappointment (and recriminations by the baseball establishment) when the Minnesota Twins defeated the A’s in the AL… Read more ›