Coming to Terms with Sport Analytics

0005

Introduction

Earlier this month, I read Richard Whittall’s discussion of Sports Analytics as a mass movement with no name. Shortly afterwards, I found Seth Partnow’s Moreyball, Goodhart’s Law and the limits of analytics. I have just finished reading another Richard Whittall post, Why Football Analysts Should Think Like Bettors.
A chance encounter in a Bowral coffee shop with an old (second) edition of Richard Lipsey’s An Introduction to Positive Economics brought my three analytics reads into focus.

A movement with no name?

I like meta posts and thought Richard’s discussion of the analytics environment as a movement with no name was fascinating.
He starts with this observation:

Sports analytics as a discipline has branded itself all wrong, and it may be too late to turn things around.

and follows up with:

I believe sports analytics is one small component of an expansive movement in the early 21st century, one I think eludes us because we’ve yet to give it a name.

This movement, Richard argues, is giving us “greater awareness that what we once considered to be ‘rational’ behavior and judgment is itself often hopelessly biased or distorted”.
This leads Richard to propose:

The problem in sports, I think, is that we’ve confused the tools we use to understand what we think we know but don’t with the reason we use them. (Original emphasis)

His post concludes with three challenges facing soccer analytics:

  1. Some clubs won’t accept what analysts are selling no matter how it is packaged or explained.
  2. What if our predictive models are dangerously flawed in ways we haven’t fully understood?
  3. How to position analytics in a much wider epistemological domain? (“It’s not just about math; it’s also about cognitive biases, about inefficiency, about hubris, self-doubt, about empiricism vs emotion.’)

Richard encourages us to expand our story to include other stakeholders:

If we can ever get past the numbers cliche, then maybe we’ll just see what’s going on in football for what it is, and truly help those teams who want to succeed in the long term.

Goodhart

I aspire to a polymath interest in the observation and analysis of performance in sport. My first degree was in social sciences (at the University of York). Part of that degree included a foundation year in Economics and Economic History.
Despite this polymath interest I am finding it difficult to keep up with some of the remarkable writing available. A Seth Partnow post reminded me about the gap between my aspirations and performance. He invoked Goodhart’s Law in his discussion of analytics.
Charles Goodhart, an economist, presented two papers in 1975 that form the basis of what subsequently has been called Goodhart’s Law, namely “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes” (Chrystal and Mizen, 2001).
Seth links basketball performance to Goodhart’s Law:

Basketball’s data era is here to stay. From optical tracking and on-demand video libraries for on-court play to wearable tech and machine-learning-based training and maintenance regimens off the floor, there has never been more information available to teams, coaches, and players. And in this environment, there is a genuine advantage to be had in being first—except when it leads to erroneous implementation of overblown or premature conclusions.

In doing so he raises fundamental issues evident in Richard’s paper. Seth’s conclusion resonates with Richard’s too:

Ultimately, the goal is to score the most points on a per possession basis, not to hit benchmarks for the sake of doing so. It’s good to play the right way, but better to play the best way for your own personnel. Striking that balance isn’t easy, but when hit, it takes analytics from the realm of the spreadsheet and puts it back on the court, where it ultimately belongs.

Positive economics analytics

I live in a town that was part of the gold rush days in New South Wales. My home in Braidwood is a former Bank used by Chinese prospectors to send money home. People came from all over the world to seek their fortunes here.
Some of the hype about analytics resonates with a gold rush. Some prospectors became very wealthy, others bought claims with no real evidence of gold-bearing potential. I take this to be the essence of Richard and Seth’s critiques. In both contexts, sport and gold mining, knowledge of the lie of the land and the local ecology are profoundly important.
This is where I find Richard Lipsey helpful. I saw a copy of the second edition in my Bowral cafe. I have made notes from the eighth edition (1995).
Richard observes:

The claim that economics is scientific stands or falls on the ability of economics to understand and predict events in the real world by stating theories, subjecting the theories to the test of real-world observations and improving the theories in the light of what has been learned. (1995:26)

I think that this rigor might start to address the challenges Richard Whittall identifies (specifically those listed as 2 and 3).
Such a positive approach to analytics would make it abundantly clear that those undertaking analytics are part of a team of people observing and analysing performance as dynamical systems in which there are Goodhart issues to address. Such an approach would have to contemplate the kind of issues raised by Mitch Mooney


And perhaps reflect this kind of interaction:


Richard Lipsey proposed that a theoretical approach requires:

  • A set of definitions that clearly describe the variables to be used.
  • A set of assumptions about the behaviour of these variables, and outlining the conditions under which the theory is to apply.
  • A set of predictions that are deduced from the assumptions of the theory, and a set of tests against actual data, to which the predictions can be subjected.

His model for the articulation of these four characteristics is:
IMG_3364
This is a model from 1995. What strikes me is that it can be part of a conversation we must have about the epistemological and ontological status of sport analytics.
I see it as a heuristic for our conversations just as his description of the process is:
IMG_3365
My reflections on the positive economics insights offered by Richard Lipsey is that we can, as a community of practice, discuss the name we give to our activities that goes beyond omnipresent and often superficial references to Moneyball. We can address the iterative implications of our observations and analyses. And we can do so in a world of growing sophistication in approaches to performance.

Performance Analysis, Performance Analytics?

I have regarded myself as a performance analyst for many years.
I admit to being confused now in a digital age of remarkable insights into the analysis of performance.
I do think that the performance analyst within me addresses the concerns Richard Whittall and Seth have about the hollowness of a purely mathematical approach to understanding performance. I seek to integrate and bring an approach to performance that has coaching and athlete learning at the heart of what I do.
But each day I do see the demonstration of enormous expertise in informatics and analytics in the sport domain.
I wonder if I am starting to work to find a new term for what I do. As a performance analyst in the latter part of the 2010s, it is unthinkable that a knowledge of analytics would not be central to what I do as I seek to contribute to the flourishing of performance.
4376565258_312cb26597_o
For my part, I see this integrating role taking place in convivial conversations with coaches and athletes in boot rooms, coffee shops and midnight diners. My world is a Sport 3.0 and an aspiring Sport 4.0 space that I am keen to share.
Whatever my new term is for my activities, I am mindful of addressing
Chris Anderson’s (2014) definition of sports analytics as:

The discovery, communication, and implementation of actionable insights derived from structured information in order to improve the quality of decisions and performance in an organization.

I do wish to add “unstructured information” into this definition too. Part of my evolving skill set is an understanding how machine learning and pattern recognition will support our work with coaches and athletes.
I see my role as making all this invisible in conversations with coaches and in service to sport.
Like Richard Whittall, I think we are in a different kind of space now that does have some profound personal learning and dispositions to address.
We have gone through the first flush of analytics in sport. We can do much better as a community of practice to transform our understanding of performance through self-reflection.
I am delighted that Seth introduced Charles Goodhart into our conversations.
All in all quite an opportunity to reflect over coffee in the town that hosts the Donald Bradman museum.

Photo Credits

Elephant Boy Cafe
Plates from Positive Economics. Richard Lipsey and Alex Chrystal (Eighth edition. Oxford: Oxford University Press, 1995:36)

RELATED ARTICLESMORE FROM AUTHOR

1 COMMENT

Leave a Reply to The PE Playbook – April 2016 Edition – drowningintheshallow Cancel reply

Please enter your comment!
Please enter your name here