The Visual Analytic Turn

Seventeen years ago, Usama Fayyad, Gregory Piatesky-Shapiro and Padhraic Smyth wrote:

Across a wide variety of fields, data are being collected and accumulated at a dramatic pace. There is an urgent need for a new generation of computational theories and tools to assist humans in extracting useful information (knowledge) from the rapidly growing volumes of digital data. These theories and tools are the subject of the emerging field of knowledge discovery in databases (KDD).

I revisited their article in the AI Magazine this week after a number of finds prompted me to think about the visual analytic turn in sport.

The first visualisation that grabbed my attention was an English Premier League fixture strength table prepared by Neil Kellie (shared with me by Julian Zipparo). Neil used Tableau Public for his visualisation.


Neil developed his table by using a static star rating and a form rating combined to give a score for each fixture. This becomes a dynamic table as the season progresses. It has prompted me to think about how we weight previous year’s ranking in a model.

The Economist added its weight to the Fantasy Football discussions with its post on 16 August. The post uses topological data-analysis software provided by Ayasdi to visualise Opta data on the different attributes of players. In an experimental interactive chart:

the data is divided into overlapping groups. These groups contain clusters of data—in this case footballers with similar attributes—which are visualised as nodes. Because the groups overlap, footballers can appear in more than one node; when they do, a branch is drawn between the nodes. Some nodes have multiple connections, whereas others have few or none.


There is a 2m 32s introduction to the Ayasdi Viewer on YouTube. Lum et al (2013) exemplify their discussion of topology with an analysis of NBA roles. Their insights received considerable publicity earlier this year (“this topological network suggests a much finer stratification of players into thirteen positions rather than the traditional division into five positions”).

Back at Tableau Public, I found news of a Fanalytics seminar. One of the presenters at the workshop is Adam McCann.  Adam’s most recent blog post is a comparison of radar and parallel coordinate charts. Adam led me to a keynote address by Noah Iliinsky: Four Pillars of Data Visualization (46m YouTube video). Noah works in IBM’s Center for Advanced Visualization.


This snowball sample underscores for me just how many remarkable people are in the visualisation space. I am interested to learn that a number of these people are using Tableau Public … to share sport data.

In other links this week, Satyam Mukherjee shared his visualisation of Batting Partnerships in the first Ashes Test 2013:




Simon Gleave’s 26 Predictions: English Premier League forecasting laid bare reminded me of the discussions following Nate Silver’s analysis of the 2012 Presidential Elections. I enjoyed Simon’s juxtaposition of 26 pre-season Premier League predictions, “13 which are at least partially model based, and 13 from the media. The models select Manchester City as title favourites but the journalists favour Chelsea”. Simon’s post introduced me to James Grayson and his reflection on predictions about performance. I think Simon and James have a very impressive approach to data.

This week’s links have left me thinking about an idea I had back in 2005. I wondered at that time if I could become skilful enough to combine the insights offered by Edward Tufte and Usama Fayyad. More recently, I have been wondering if I could do that with the virtuosity that pervades Snow Fall.

Openness and Curiosity: Oceans Apart

Yesterday my wife Sue received an alert to an Economist blog post about James Heckman.

James had given a talk at the Centre for Economic Performance that was co-hosted by the Young Foundation and the Studio Schools Trust.

The title of James’ talk was Creating a More Equal and Productive Britain.

The summary note of his talk records that Professor Heckman discussed that openness and curiosity have a greater effect on academic outcomes than IQ scores. He argued that:

contrary to long-held assumptions that these skills cannot be measured – the evidence is available and should be incorporated much more actively into spending decisions –  particularly in times of cuts.

I liked the case James made for early investment in these skills, “through character education in schools targeted primarily at less advantaged pupils and proactive family policy”.

These character skills are:

Openness (curiosity, willing to learn)

Consciousness (staying on task)

Extroversion (outgoing, friendly)

Agreeableness (helpful)

Neuroticism (attention to detail, persistence).

Madeleine Bunting observes that:

These are the skills that enable children to learn; without them even the best teachers can do little. These are the skills that are predictive of outcomes such as educational achievement, obesity, offender rates, employment and smoking. The single biggest predictor of longevity and school achievement is conscientiousness – which is effectively a form of self-control.

James’ audio of the talk, the slides he used for his presentation and a 2011 paper on Personality Psychology and Economics are available on the Young Foundation’s website. I noted in the paper James and his co-authors observe that:

There is a small but growing body of intervention studies that establish that personality traits can be altered over long periods of time in response to interventions. Some of the major effects of early childhood intervention programs appear to operate through their lasting effects on personality. Family investment decisions also change personality.

By chance the Economist post took me to Katharine Birbalsingh‘s book. This link to Katherine’s writing came a few days after listening to Allan Brahminy.

Katherine’s website home page starts with these two paragraphs:

Katharine Birbalsingh has been teaching in the state school system in London for over a decade. Her dream is for all schools to become interesting and exciting places of learning, where children feel safe, happy and free to aim to be the best that they can be.

Children should be challenged to achieve all that they can and should be rewarded when they do. She wants the children in her care to have a sense of responsibility, to have a sense of ownership of their school, their lives, and their futures.

Allan runs a centre based in the Northern Territory, designed to rehabilitate profoundly troubled young people. “For many of these young hardened criminals and drug addicts, the Brahminy program is their last resort. They’ve been through the system and spat out the other side.” ABC Television is broadcasting a three-part series, Outback Kids, about Allan’s work.

The Brahminy Foundation’s Social Justice Statement is:

Brahminy recognizes that not all people and young people share equally in the benefits of society, and it is imperative that options exist for those most disadvantaged.

Therefore, all people and young people involved with Brahminy:

  • Will be supported without discrimination.
  • Should be treated with respect so that their dignity as individuals is preserved.
  • Should be recognized as people capable of making decisions and choices for their own lives.

I understand that James, Katherine and Allan have different approaches to “meaningful difference” and that there are contentious issues around Katherine and Allan’s work.

However all three have helped me reflect on James’s points that:

  • Adversity gets under the skin and determines the biology of the child
  • Society must supplement the parenting resources of troubled families
  • Prevention not remediation

James’ concluding slide indicates just how early the work must start:

I am keen to ensure that children’s learning is playful too. Recent research reports suggest that inactivity is having an enormous impact on wellbeing.

Photo Credit

Family picnicking under a tree

QR Code for this post

On Being Social … and Well

This post started with a conversation about an RSS alert from The Economist.

My wife, Sue, receives a range of RSS feeds each day on her iPad. Earlier this week a story about social isolation and illness led to a fascinating breakfast time and on-going conversation.

It prompted me to follow up on the paper that informed The Economist article.

Julianne Holt-Lunstad, Timothy Smith and Bradley Layton’s paper Social Relationships and Mortality Risk: A Meta-analytic Review appeared in PLoS Medicine, July 2010. Its publication in PLoS Medicine made the paper even more interesting for me given the open access nature of the journal.

The objectives of the paper, a meta-analytic review, were ” to determine the extent to which social relationships influence risk for mortality, which aspects of social relationships are most highly predictive, and which factors may moderate the risk.” Julianne, Timothy and Bradley reviewed 148 studies (from January 1900 to January 2007) involving a total of 308,849 participants and suggest that “people with stronger social relationships had a 50% increased likelihood of survival than those with weaker social relationships.”

Their summary of their findings is:

These findings indicate that the influence of social relationships on the risk of death are comparable with well-established risk factors for mortality such as smoking and alcohol consumption and exceed the influence of other risk factors such as physical inactivity and obesity. Furthermore, the overall effect of social relationships on mortality reported in this meta-analysis might be an underestimate, because many of the studies used simple single-item measures of social isolation rather than a complex measurement. Although further research is needed to determine exactly how social relationships can be used to reduce mortality risk, physicians, health professionals, educators, and the media should now acknowledge that social relationships influence the health outcomes of adults and should take social relationships as seriously as other risk factors that affect mortality, the researchers conclude.

These are the metrics for the paper at the time I accessed it:

I wonder if this reader profile is a good example of a social effect too.

Greg Miller has noted subsequently that:

In a steady stream of recent papers, social psychologists have identified several potentially unhealthy changes in the cardiovascular, immune, and nervous systems of chronically lonely people. The findings could help explain why epidemiological studies have often found that socially isolated people have shorter life spans and increased risk of a host of health problems, including infections, heart disease, and depression. The work also adds a new wrinkle, suggesting that it’s the subjective experience of loneliness that’s harmful, not the actual number of social contacts a person has. An impressive network of collaborations with researchers in other disciplines is now pioneering a new science of loneliness.

Stephen Cole has added to the discussion with his paper at AAAS in February 2011. He and a number of colleagues have published a paper in Genome Biology too. They conclude in that paper:

These data provide the first indication that human genome-wide transcriptional activity is altered in association with a social epidemiological risk factor. Impaired transcription of glucocorticoid response genes and increased activity of pro-inflammatory transcription control pathways provide a functional genomic explanation for elevated risk of inflammatory disease in individuals who experience chronically high levels of subjective social isolation.

Many years ago as a young student I read about anomie and alienation.  A breakfast conversation and a search for links has brought back memories of these early readings of Durkheim and Marx. They were crystallised for me by a conversation in our family car about Albert Camus’ The Outsider on a road trip in Tasmania this week.

I realise how fortunate I am to have the opportunity for breakfast conversations and family car journeys.

Photo Credit