The Visual Analytic Turn

Seventeen years ago, Usama Fayyad, Gregory Piatesky-Shapiro and Padhraic Smyth wrote:

Across a wide variety of fields, data are being collected and accumulated at a dramatic pace. There is an urgent need for a new generation of computational theories and tools to assist humans in extracting useful information (knowledge) from the rapidly growing volumes of digital data. These theories and tools are the subject of the emerging field of knowledge discovery in databases (KDD).

I revisited their article in the AI Magazine this week after a number of finds prompted me to think about the visual analytic turn in sport.

The first visualisation that grabbed my attention was an English Premier League fixture strength table prepared by Neil Kellie (shared with me by Julian Zipparo). Neil used Tableau Public for his visualisation.


Neil developed his table by using a static star rating and a form rating combined to give a score for each fixture. This becomes a dynamic table as the season progresses. It has prompted me to think about how we weight previous year’s ranking in a model.

The Economist added its weight to the Fantasy Football discussions with its post on 16 August. The post uses topological data-analysis software provided by Ayasdi to visualise Opta data on the different attributes of players. In an experimental interactive chart:

the data is divided into overlapping groups. These groups contain clusters of data—in this case footballers with similar attributes—which are visualised as nodes. Because the groups overlap, footballers can appear in more than one node; when they do, a branch is drawn between the nodes. Some nodes have multiple connections, whereas others have few or none.


There is a 2m 32s introduction to the Ayasdi Viewer on YouTube. Lum et al (2013) exemplify their discussion of topology with an analysis of NBA roles. Their insights received considerable publicity earlier this year (“this topological network suggests a much finer stratification of players into thirteen positions rather than the traditional division into five positions”).

Back at Tableau Public, I found news of a Fanalytics seminar. One of the presenters at the workshop is Adam McCann.  Adam’s most recent blog post is a comparison of radar and parallel coordinate charts. Adam led me to a keynote address by Noah Iliinsky: Four Pillars of Data Visualization (46m YouTube video). Noah works in IBM’s Center for Advanced Visualization.


This snowball sample underscores for me just how many remarkable people are in the visualisation space. I am interested to learn that a number of these people are using Tableau Public … to share sport data.

In other links this week, Satyam Mukherjee shared his visualisation of Batting Partnerships in the first Ashes Test 2013:




Simon Gleave’s 26 Predictions: English Premier League forecasting laid bare reminded me of the discussions following Nate Silver’s analysis of the 2012 Presidential Elections. I enjoyed Simon’s juxtaposition of 26 pre-season Premier League predictions, “13 which are at least partially model based, and 13 from the media. The models select Manchester City as title favourites but the journalists favour Chelsea”. Simon’s post introduced me to James Grayson and his reflection on predictions about performance. I think Simon and James have a very impressive approach to data.

This week’s links have left me thinking about an idea I had back in 2005. I wondered at that time if I could become skilful enough to combine the insights offered by Edward Tufte and Usama Fayyad. More recently, I have been wondering if I could do that with the virtuosity that pervades Snow Fall.

Growing Sport Analytics

I have been thinking a great deal about transforming performance and leading ahead of the curve this year. By coincidence this year is ending with the screening of Moneyball.

I have been interested in particular in how secondary data can inform and support coaches and the coaching process.

One of the catalysts in my thinking has been Usama Fayyad. I heard Usama speak at a knowledge discovery in databases conference in Sydney in 2005. His 1996 paper From Data Mining to Knowledge Discovery in Databases co-written with Gregory Piatetsky-Shapiro and Padhraic Smyth was my first engagement with a domain of enquiry that has become a primary focus for me.

In their 1996 paper the authors point out that:

Across a wide variety of fields, data are being collected and accumulated at a dramatic pace. There is an urgent need for a new generation of computational theo- ries and tools to assist humans in extracting useful information (knowledge) from the rapidly growing volumes of digital data. These theories and tools are the subject of the emerging field of knowledge discovery in databases (KDD).

I spent much of the 1980s and 90s collecting data about performance in rugby union and a number of other sports. All of these data were collected with hand notation systems. My interest then and now is the pattern of observable individual and team behaviour.

I saw an early copy of Michael Lewis’s book The Art of Winning an Unfair Game (2003) and was attracted intuitively to the power of sabermetrics. I saw in Bill James’s work the passion that fired Charles Reep in his observations of association football. Michael, Bill and Charles had a predecessor in Hugh Fullerton who in 1910 wrote about The Inside Game of baseball.

In his paper, Hugh Fullerton points out:

Last season (1909) I arranged with scorers to record hits of various kinds, and secured the scores thus kept on 40 Central League games, 26 American Association games, and fourteen college games to compare with major league scores kept in the same manner. In the college games one grounder in every 8 1/3 passed the infielders. In the Central League one in 10 7/12, in the American Association one in 12 2/43, and in the American National Leagues (45 games of my own scoring) one in every 15 3/16.

He adds that:

The figures were amazing, as they followed so closely the classification of the leagues. They proved that there is a reason for the “class”, but the proof is not found in the mathematics, but in two word (unless you hyphenate them), “team work.”

For Hugh Fullerton the inside game is “the art of getting the hits that “he couldn’t have got anyhow””.

102 years on from Hugh Fullerton’s research there is a growing community of practice in sport analytics. A recent example of this incandescence was a Sports Analytics conference in Manchester in November.

Speakers at the conference included: Bill Gerrard, Raffaele Poli, Simon Wilson, David Fallows, Gavin Fleig, Ed Sulley, Ian Lenagan, Rob Lowe, Fergus Connolly, Ian Graham, Nick Broad and Steve Houston.

Usama Fayyad, Gregory Piatetsky-Shapiro and Padhraic Smyth  point out that Knowledge KDD is “the nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data”.

I believe they have provided a fundamental guide to the family resemblances that characterise the analysis of sport performance:

Here, data are a set of facts (for example, cases in a database), and pattern is an expression in some language describing a subset of the data or a model applicable to the subset. Hence, in our usage here, extracting a pattern also designates fitting a model to data; finding structure from data; or, in general, making any high-level description of a set of data. The term process implies that KDD comprises many steps, which involve data preparation, search for patterns, knowledge evaluation, and refinement, all repeated in multiple iterations. By nontrivial, we mean that some search or inference is involved; that is, it is not a straightforward computation of predefined quantities like computing the average value of a set of numbers.

I am hopeful that 2012 will provide opportunities to share data throughout the community of practice that is sport analytics.

Photo Credits


Australian bowler, Bill O’Reilly, demonstrates his famous grip