A letter to the Secret Soccer Analyst

A training session

Dear Secret Soccer Analyst

I have four apologies to make to you.

First, I am sorry I missed your post when you first published it. Fortunately, Richard Whittall, Rob Carroll and Darrell Cobner alerted me to your story.

When I last looked Richard’s tweet had received 918 likes and 255 retweets. Martin Bucheit’s tweet about your post has 19 likes.

Your open sharing has touched many people. I am late to the party but am touched too. Profoundly so.

A picture of a videographer at a training session.

 I have been involved in the analysis of performance since the late 1970s.

In the 1990s, I worked as a performance analyst in rugby union. For most of the decade, I had intense work periods that during competition and overseas tours extended to 18 hour days … week after week.

It was an analogue video world. I was fortunate to have access to host broadcast videos and captured my own game video with the help of colleagues. We thought we had hit the jackpot with S-VHS format.

I have a second apology to make.

When I started the Centre for Notational Analysis in Cardiff in 1991, I hoped to create a career path for young people interested in performance analysis. We were at the start of a movement that has positioned you to feel the way you do.

I am staggered by the creativity of people like yourself. The world you occupy is a quantum shift from the occupational culture of the 1990s.

We do share the same vision though … to provide an invisible service to coaches that records, analyses and potentially transforms performance.

Filming training

My third apology is that despite my advocacy for performance analysis as a service, I have had limited success in persuading clubs and sporting organisations that our work should be valued rather than priced.

Young people are still inducted into performance analysis as unpaid interns or lowly paid analysts. We are encouraged to believe that the status of being connected with a club or team ‘compensates’ for zero or low wages.

Some learning organisations are pro-actice and make every effort to support their analysts financially and through continuing learning opportunities.

Our dilemma is that we have so many people wanting to be analysts and often have the experiences you have.

My fourth apology is to lament that I have not been able to gain acceptance for two fundamental questions:

What is so important we have to share it?

Can less be more?

At the heart of this conversation is a profound debate about frequent augmented information. The availability of hardware and software has led to a surfeit of options for each analyst.

It has created an insatiability that we as a profession must address. We either need more people or automation to help us or we use the two questions above and add two more:

What has changed in performance in training and competition?

How might feedforward transform our practice?

Much of my time now is spent in exhorting our community of practice to share experiences openly. Your post is a very important marker in our conversation about practice and reminded me of that quote in Shadowlands

We read to know we are not alone.

Now that you have shared your thoughts, I hope you realise that you have a world of friends who are just a phone call or email away.

I hope too you will accept my apologies.

Keith

Photo Credits

Training day in 38C heat (Keith Lyons, CC BY 4.0)

Coming to Terms with Sport Analytics

0005

Introduction

Earlier this month, I read Richard Whittall’s discussion of Sports Analytics as a mass movement with no name. Shortly afterwards, I found Seth Partnow’s Moreyball, Goodhart’s Law and the limits of analytics. I have just finished reading another Richard Whittall post, Why Football Analysts Should Think Like Bettors.

A chance encounter in a Bowral coffee shop with an old (second) edition of Richard Lipsey’s An Introduction to Positive Economics brought my three analytics reads into focus.

A movement with no name?

I like meta posts and thought Richard’s discussion of the analytics environment as a movement with no name was fascinating.

He starts with this observation:

Sports analytics as a discipline has branded itself all wrong, and it may be too late to turn things around.

and follows up with:

I believe sports analytics is one small component of an expansive movement in the early 21st century, one I think eludes us because we’ve yet to give it a name.

This movement, Richard argues, is giving us “greater awareness that what we once considered to be ‘rational’ behavior and judgment is itself often hopelessly biased or distorted”.

This leads Richard to propose:

The problem in sports, I think, is that we’ve confused the tools we use to understand what we think we know but don’t with the reason we use them. (Original emphasis)

His post concludes with three challenges facing soccer analytics:

  1. Some clubs won’t accept what analysts are selling no matter how it is packaged or explained.
  2. What if our predictive models are dangerously flawed in ways we haven’t fully understood?
  3. How to position analytics in a much wider epistemological domain? (“It’s not just about math; it’s also about cognitive biases, about inefficiency, about hubris, self-doubt, about empiricism vs emotion.’)

Richard encourages us to expand our story to include other stakeholders:

If we can ever get past the numbers cliche, then maybe we’ll just see what’s going on in football for what it is, and truly help those teams who want to succeed in the long term.

Goodhart

I aspire to a polymath interest in the observation and analysis of performance in sport. My first degree was in social sciences (at the University of York). Part of that degree included a foundation year in Economics and Economic History.

Despite this polymath interest I am finding it difficult to keep up with some of the remarkable writing available. A Seth Partnow post reminded me about the gap between my aspirations and performance. He invoked Goodhart’s Law in his discussion of analytics.

Charles Goodhart, an economist, presented two papers in 1975 that form the basis of what subsequently has been called Goodhart’s Law, namely “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes” (Chrystal and Mizen, 2001).

Seth links basketball performance to Goodhart’s Law:

Basketball’s data era is here to stay. From optical tracking and on-demand video libraries for on-court play to wearable tech and machine-learning-based training and maintenance regimens off the floor, there has never been more information available to teams, coaches, and players. And in this environment, there is a genuine advantage to be had in being first—except when it leads to erroneous implementation of overblown or premature conclusions.

In doing so he raises fundamental issues evident in Richard’s paper. Seth’s conclusion resonates with Richard’s too:

Ultimately, the goal is to score the most points on a per possession basis, not to hit benchmarks for the sake of doing so. It’s good to play the right way, but better to play the best way for your own personnel. Striking that balance isn’t easy, but when hit, it takes analytics from the realm of the spreadsheet and puts it back on the court, where it ultimately belongs.

Positive economics analytics

I live in a town that was part of the gold rush days in New South Wales. My home in Braidwood is a former Bank used by Chinese prospectors to send money home. People came from all over the world to seek their fortunes here.

Some of the hype about analytics resonates with a gold rush. Some prospectors became very wealthy, others bought claims with no real evidence of gold-bearing potential. I take this to be the essence of Richard and Seth’s critiques. In both contexts, sport and gold mining, knowledge of the lie of the land and the local ecology are profoundly important.

This is where I find Richard Lipsey helpful. I saw a copy of the second edition in my Bowral cafe. I have made notes from the eighth edition (1995).

Richard observes:

The claim that economics is scientific stands or falls on the ability of economics to understand and predict events in the real world by stating theories, subjecting the theories to the test of real-world observations and improving the theories in the light of what has been learned. (1995:26)

I think that this rigor might start to address the challenges Richard Whittall identifies (specifically those listed as 2 and 3).

Such a positive approach to analytics would make it abundantly clear that those undertaking analytics are part of a team of people observing and analysing performance as dynamical systems in which there are Goodhart issues to address. Such an approach would have to contemplate the kind of issues raised by Mitch Mooney

And perhaps reflect this kind of interaction:

Richard Lipsey proposed that a theoretical approach requires:

  • A set of definitions that clearly describe the variables to be used.
  • A set of assumptions about the behaviour of these variables, and outlining the conditions under which the theory is to apply.
  • A set of predictions that are deduced from the assumptions of the theory, and a set of tests against actual data, to which the predictions can be subjected.

His model for the articulation of these four characteristics is:

IMG_3364

This is a model from 1995. What strikes me is that it can be part of a conversation we must have about the epistemological and ontological status of sport analytics.

I see it as a heuristic for our conversations just as his description of the process is:

IMG_3365

My reflections on the positive economics insights offered by Richard Lipsey is that we can, as a community of practice, discuss the name we give to our activities that goes beyond omnipresent and often superficial references to Moneyball. We can address the iterative implications of our observations and analyses. And we can do so in a world of growing sophistication in approaches to performance.

Performance Analysis, Performance Analytics?

I have regarded myself as a performance analyst for many years.

I admit to being confused now in a digital age of remarkable insights into the analysis of performance.

I do think that the performance analyst within me addresses the concerns Richard Whittall and Seth have about the hollowness of a purely mathematical approach to understanding performance. I seek to integrate and bring an approach to performance that has coaching and athlete learning at the heart of what I do.

But each day I do see the demonstration of enormous expertise in informatics and analytics in the sport domain.

I wonder if I am starting to work to find a new term for what I do. As a performance analyst in the latter part of the 2010s, it is unthinkable that a knowledge of analytics would not be central to what I do as I seek to contribute to the flourishing of performance.

4376565258_312cb26597_o

For my part, I see this integrating role taking place in convivial conversations with coaches and athletes in boot rooms, coffee shops and midnight diners. My world is a Sport 3.0 and an aspiring Sport 4.0 space that I am keen to share.

Whatever my new term is for my activities, I am mindful of addressing
Chris Anderson’s (2014) definition of sports analytics as:

The discovery, communication, and implementation of actionable insights derived from structured information in order to improve the quality of decisions and performance in an organization.

I do wish to add “unstructured information” into this definition too. Part of my evolving skill set is an understanding how machine learning and pattern recognition will support our work with coaches and athletes.

I see my role as making all this invisible in conversations with coaches and in service to sport.

Like Richard Whittall, I think we are in a different kind of space now that does have some profound personal learning and dispositions to address.

We have gone through the first flush of analytics in sport. We can do much better as a community of practice to transform our understanding of performance through self-reflection.

I am delighted that Seth introduced Charles Goodhart into our conversations.

All in all quite an opportunity to reflect over coffee in the town that hosts the Donald Bradman museum.

Photo Credits

Elephant Boy Cafe

Plates from Positive Economics. Richard Lipsey and Alex Chrystal (Eighth edition. Oxford: Oxford University Press, 1995:36)