What Counts?

Introduction

I have been thinking about Darrell Cobner’s most recent post (13 September) on the Visual Performance Analysis blog.
In the post Darrell asks “Any performance analysts actually use statistical tests in the applied world?”
My short answer is that I have not … but I understand that a rich, deep back office support for coaches and athletes is transformed by statistical insight. This back office helps with the pressing needs of front office performance described well by Will Oldham this week.
I think statistical insight can ensure that the urgent demands placed upon analysts each day do not lead us to forget longer-term transformation of performance. I do believe that these insights need to be crafted into stories. I tried to outline an approach to sharing performance stories in this presentation in Porto in 1998.

Describing

I have spent most of the last thirty years pursuing descriptive-analytic approaches to the systematic observation of performance. I have only ever used descriptive statistics in my work and have admired from afar the skills of people like Will Hopkins, Alan Nevill and Steve Cooper who see the world from a completely different perspective.
My epistemological guides to research have included William Anderson, David Berliner, Elliot Eisner, Lawrence Stenhouse and Norman Denzin. When it came to re-presenting research to a range of audiences I took stock of what John Van Maanen, Donald Polkinghorne and Miller Mair had to say about story telling.
I delighted in Peter Reason and John Rowan’s Human Inquiry when it was first published in 1981. I was profoundly smitten by the qualitative turn and have spent much of my time since then thinking about and trying to address profound issues of validity and reliability in observation and sharing.
My 1998 presentation did try to look at the theory and practice of sharing and at that time.

I was keen to share Donald Polkinghorne’s ideas about narrative knowing and Clifford Geertz’s thick description.

Darrell’s post has brought these framing issues back to me.

What Counts?

Darrell’s post has come at an excellent time for me.
I have been thinking a great deal about the occupational culture of performance analysts of late.
I am fascinated by the Visual Analytic Turn in what we do. In my own work I have become greatly interested in colour and how it might re-present performance.
I hope that all my work is still founded on disciplined, systematic observation and ever mindful of Donald Campbell and Julian Stanley’s research design exhortations.
I am conscious that in my approach I do not use random sampling. I am a purposive sampler. I do not talk about my work being representative but I do hope whatever I share resonates with others. I have tried to share these constraints openly.
Whilst reflecting on Darrell’s post, I was reading through a paper prepared by one of my PhD students. The student has coached elite football for over thirty years and has spent the last three decades providing analysis of performance data to international teams. This student has come to statistical insight late in his career but is now fascinated by how it can support the face validity of his practice as an analysts and coach.
I was delighted to read this paragraph in his work:

The data for categories and regained possessions were assessed for normality, using Kolmogorov –Smirnov and Shapiro-Wilk tests (0.162, 9df, Sig 0.2) (0.947, 9df, Sig 0.657) and (0.213,9df, Sig. 0.2) (0.908,9df, Sig 0.299) respectively.  Levene’s test for equal variance statistic and Sig. score was (0.151, 0.863) and (1.624, 0.273) respectively, confirming suitability for parametric tests.

and this:

Yule’s Q test was used to test inter operator reliability in coding each category of goals in Open Play, with scores of 0.993, 0.986 and 0.968 for category (a), (b) and (c) respectively.

I had the opportunity to act as a reviewer for another paper this week. It reported the close working relationship between a statistician and two triathlon coaches investigating performance profiles over a macro-Olympic cycle. The paper uses some sophisticated statistical tests but I enjoyed the sense the coaches made of the data. Their narrative was enriched by the statistical insights.
The emergence of a vibrant knowledge discovery community in sport, the increasing use of algorithms and powerful visualisations suggest to me that performance analysis is wonderfully placed to deliver agnostic support for coaches and athletes. We have analysts who use proprietary software like JMP and open source tools like R. A few months ago I wrote about how impressed I was with developments in New Zealand.
Each of us makes choices about how we observe and analyse data. I am keen to explore how we share these observations and analyses as story tellers.
I am more and more impressed by young analysts who are able to blend a whole range of approaches to analysis that include statistical insight. Earlier this year, I enjoyed learning about the work an embedded PhD student conducted at an AFL club to monitor training load and intensity. I was impressed by the mediation analysis approach used. This young researcher now works in an Institute of Sport and is working with coaches to plan, monitor and evaluate training. His statistical insights are proving very valuable to coaches.
Two months ago I watched a young undergraduate intern sitting in a county cricket pavilion working with an assistant coach exploring some probability approaches to scoring patterns in Twenty Twenty cricket. It seemed like a perfect teachable moment for the intern and the coach.
This section has been a longer response to Darrell’s question. I am hopeful that in the stories analysts share there is an understanding of the principles of statistical insight and the potential to blend a powerful narrative.
For my part, I will be continuing with my lifelong interest in the poetics of performance.

Poetics of Performance

Miller Mair died in 2011. In his obituary, David Small wrote of Miller:

the intellectual independence of his extensive publications gives his ideas an appeal beyond that of any particular school. Principally this grew from a fascination with language and the avenues for therapeutic communication it offered. His 1989 book Between Psychology and Psychotherapy was subtitled “a poetics of experience”, and this theme recurs throughout his written and spoken work. He saw therapist and client as reaching towards understanding through conversation and metaphor, through engaging with the “community of selves” of which they were personally constituted, and through striving to “tell stories” that would illuminate the conditions of their lives.

Photo Credit

Miller Mair (CPN Niagara Conference, 2010)