Basketball: archives and insights

On 19 December 2017, Google Cloud announced that it had become the official cloud partner for the National Collegiate Athletic Association (NCAA).

In the announcement, it was reported that:

the NCAA is migrating 80+ years of historical and play-by-play data, from 90 championships and 24 sports, to Google Cloud Platform

One of the first activities planned was to explore basketball data in preparation for the NCAA’s Women’s and Men’s Division I Basketball Tournaments held in March and April 2018 (March Madness).

More information about the partnership appeared in two posts on 30 March 2018. In the first post, Courtney Blacker reported a month’s-long experiment “to apply Google’s technologies to the NCAA’s treasure trove of data”. 

We assembled a team of technicians, data scientists, and basketball enthusiasts (we call them ‘The Wolfpack’) who built a data processing workflow using Google Cloud Platform technologies like BigQuery and Cloud Datalab.

The aim of this approach was “to build models that look at influential factors on team performance”. During the tournaments, the Google Cloud team planned to “use our workflow to analyze our observations from the first half of each game against NCAA historical data to hone in on a stat-based prediction for the second half that we think is highly probable”. These predictions would be presented as a television advert during the half time break.

An example from the Kansas v Villanova semi-final game:

The video suggested there would be at least 26 assists in the second half (there were 28) and 55 shot attempts (there were 64). (In the second semi final, Michigan v Loyola-Chicago, the predictions were for 37 three-point attempts (there were 38) and 29 rebounds (there were 29).

The final had these suggestions:

The second post, written by Eric Schmidt and Allen Jarvis,  about the Google Cloud and NCAA partnership, provided a detailed account of the architecture to support the data analysis that was occurring. This illustrated “the importance of proper tooling to enable collaboration across multiple disciplines, including data engineering, data analysis, data science, quantitative analysis, and machine learning”.

The architecture for this service requires:

  1. A flexible and scalable data processing workflow to support collaborative data analysis.
  2. New analytic explorations through collaboratively developed queries and visualizations.
  3. Real-time predictive insights and analysis related to the games, modeled around NCAA men’s and women’s basketball.

Eric and Allen go through each of these points at length. Their account indicates what is becoming available to sport as we explore archives for insights.

They have an important message in their conclusion:

… better data preparation means better data analysis. Many organizations imagine diving in directly to predictive modeling without a critical examination of their data or existing analytic frameworks. If the greatest value is to be found in predictive insights, followed by analysis, supported by clean but raw data, you can imagine the amount of work required to get there as the inverse: a lot of data preparation that paves the way for better analysis, which in turn clears a path for good modeling.

The ball is in all our courts.

Photo Credits

March Madness 2009 (Andy Thrasher, public domain)

Gators are in the Final Four (Courtland, CC BY-NC-ND 2.0)

Intelligence Augmentation: meeting Vannevar and Douglas


It has taken me some time but I have managed to unearth some primary sources in the discussion of intelligence augmentation.

I do think this is a profoundly important concept to consider when we contemplate our relationship to artificial intelligence in our cultural contexts.

A slide shared by Melanie Cook (2017) sent me off on my reading journey.

Peter Skagestad (1993:157) set me on the way too. He observed:

the pioneers of the personal-computer revolution did not theorize about the essence of the computer, but focused rather on the essence of human thinking, and then sought ways to adapt computers to the goal of improving human thinking.

His discussion took me to Douglas Engelbart’s report Augmenting Human Intellect: A Conceptual Framework (October 1962) prepared for the Director of Information Sciences in the Air Force Office of Scientific Research.

Douglas introduced me to Vannevar Bush’s article As We May Think (1945).

In this post I introduce you to Vannevar and Douglas. I apologise if you have met them both already.

Vannevar Bush

There is a very detailed account of Vannevar’s life and work in Wikipedia. His 1945 article provides an introduction to his thinking and a vision for the scientific endeavour he nurtured in the next four decades. The Atlantic Editor notes “Dr. Bush calls for a new relationship between thinking man and the sum of our knowledge”.

Douglas Engelbart (1962:48) quotes Vannevar extensively: “it was deemed appropriate to our purpose here to summarize it in detail and to quote from it at considerable length”.

There are some key passages for me in Vannevar’s article. The first is to do with communication:

Science has provided the swiftest communication between individuals; it has provided a record of ideas and has enabled man to manipulate and to make extracts from that record so that knowledge evolves and endures throughout the life of a race rather than that of an individual.

And how we manage these communications:

There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialization extends. Professionally our methods of transmitting and reviewing the results of research are generations old and by now are totally inadequate for their purpose. If the aggregate time spent in writing scholarly works and in reading them could be evaluated, the ratio between these amounts of time might well be startling. The difficulty seems to be, not so much that we publish unduly in view of the extent and variety of present day interests, but rather that publication has been extended far beyond our present ability to make real use of the record.

Then there is:

The summation of human experience is being expanded at a prodigious rate, and the means we use for threading through the consequent maze to the momentarily important item is the same as was used in the days of square-rigged ships.

Vannevar notes that there is help on the horizon, namely, “there are signs of a change as new and powerful instrumentalities come into use”.Vannevar’s example of an instrumentality was an imagined device, a memex:

in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.

A memex is primarily a piece of furniture. It is designed to support the human mind’s associative inquiry (an “intricate web of trails carried by the cells of the brain”) and “any item may be caused at will to select immediately and automatically another”.

Douglas Engelbart


Seventeen years after Vannevar’s article, Douglas presented his report Augmenting Human Intellect: A Conceptual Framework to the Director of Information Sciences in the Air Force Office of Scientific Research.

Mike Cassidy (2013) wrote of Douglas:

He believed that computers, which were primarily for crunching numbers and spitting out answers when he started his work, had the ability to empower people and enhance their intellect in ways that would improve lives.

Douglas’s 1962 report addressed this empowerment. It was “an initial summary report of a project taking a new and systematic approach to improving the intellectual effectiveness of the individual human being” (1962:ii).

‘Augmenting the human intellect’ for Douglas meant increasing a person’s capability to approach a complex situation and derive solutions to the problem. This will involve providing access to “the services of a digital computer” and “developing the new methods of thinking and working” that allow the human to capitalize on these services” (1962:3). (He notes elsewhere “Individuals who operate effectively in our culture have already been considerably ‘augmented’. (1962:15))

Douglas’s report presents a conceptual framework that orients us “toward the real possibilities and problems associated with using modern technology to give direct aid to an individual in comprehending complex situations, isolating the significant factors, and solving problems” (1962:8).

He suggests that our capabilities are augmented by:

  • artifacts
  • language
  • methodologies for problem-solving
  • training

He proposes:

The system we want to improve can thus be visualized as a trained human being together with his artifacts, language, and methodology. The explicit new system we contemplate will involve as artifacts computers,and computer-controlled information-storage, information-handling, and information-display devices. (1962:9)

Douglas named this system H-LAM/T (Human using Lauguage, Artifacts, Methodology, in which he/she is Trained).

As he explored his conceptual approach to augmentation, Douglas discussed Vannevar’s memex ideas at length (“This material is so relevant and so well put that I quote it in its entirety” (1962:50).)

Douglas concludes his report with ta discussion of how the intellectual effectiveness of a human can be significantly improved by an engineering-like approach toward redesigning changeable components of a system. (1962:128)

The aim of such an engineering-like approach is to provide:

potential users in different domains of intellectual activity with a basic general-purpose augmentation system from which they themselves can construct the special features of a system to match their jobs, and their ways of working—or it could be used on the other hand by researchers who wanted to pursue the development of special augmentation systems for special fields. (1962:130)

Douglas concludes his report with advocacy for a dynamic discipline aimed at understanding and harnessing ‘neural power’.


Vannevar and Douglas’s visions for intelligence augmentation make for fascinating reading seventy-three and fifty-six years on respectively. I do think they should be essential reading for anyone exploring sport informatics and analytics and contemplating special augmentation systems for such a special field.

I hope both of them would take some delight in Ross Goodwin’s suggestion (2016):

When we teach computers to write, the computers don’t replace us any more than pianos replace pianists—in a certain way, they become our pens, and we become more than writers. We become writers of writers.

I trust that in becoming writers of writers, we create new opportunities to incorporate artificial intelligence into our own augmentation and our transformation of practice.

Photo Credits

Telephone Switchboard Operators 1914 (Reyner Media, CC BY 2.0)

Vannevar Bush (This image is a work of the United States Department of the Treasury, taken or made as part of an employee’s official duties. The image is in the public domain in the United States.)

Douglas Engelbart (Smithsonian Magazine)

On the way to the training ground (Keith Lyons, CC BY 4.0)

Thinking about analytics

A line in Colin Beer’s Learning analytics and magic beans (1 March 2018) “Learning analytics requires a learning approach …” sent me off thinking about how discussions about learning analytics in education might facilitate conversations about analytics in sport.

I produced this slide deck to think out loud:

Photo Credit
Making Sense Of The Data: For You And Your Coach (Jacquie Tran, 2014)