A Conversation with Jock Mackinlay: The Two Success Stories Driving Visual Analysis

One great thing about working about Tableau is that I get to work with smart people who have years of research experience investigating how to help people see and understand data better. Recently, Jock Mackinlay, Tableau’s Director of Visual Analysis and one of the world’s most expert information visualization scientists, and I had a chance to spend some “think time” together. We talked about a recent article he wrote on the topic of collaborative visualization.

By way of background, the article Jock wrote was a technical perspective titled "Finding and Telling Stories with Data" on an important research paper regarding collaborative visualization; the paper itself is titled "Voyagers and Voyeurs: Supporting Asynchronous Collaborative Visualization" (both pieces require website registration before you can access them). Jock's technical perspective and the paper were published in the magazine "The Communications of the Association for Computing Machinery", or the CACM, which is one of premier publications for computer professionals.

Elissa Fink (EF): Jock, tell me more about this recent article you wrote.

Jock Mackinlay (JM): Well, I was asked to write an introduction to an important paper about collaborative visualization. As part of that, I wanted to talk about Visual Analysis as a powerful method for finding and telling stories with data. The idea of telling stories with data is moving from research into widespread use where collaboration is a necessity. And it’s being driven by two success stories of computing: databases and visual interfaces.

EF: What do you mean by the “success story” of databases? I’ve never really thought of databases as being either successful or unsuccessful. And frankly, sometimes when I use databases, I feel pretty unsuccessful.

JM: Well, if you think about it, before databases, there were just big, messy files. The success of databases began in 1970 with Codd’s relational model, which supported transaction processing. And before you ask, Edgar Codd was an IBM computer scientist working in Britain. His invention – the relational database - truly revolutionized the world of business. By the 1980s, you saw the widespread reorganization of businesses, hospitals, schools, and governments to use relational databases. More recently - mostly throughout the 1990’s - due to the internet and accumulation of online data, we’ve seen the development of multi-dimensional data warehouses designed to support reporting and analytical queries. And since then, the trend has been a steady increase in the number and size of databases that support analysis.

EF: O.k. You’re right – I can see what a big impact databases have had on our world. But how do they and visual interfaces relate to story-telling?

JM: First, don’t forget about the success of visual interfaces themselves. They began in the 1960s with graphical user interfaces (GUIs), which replaced command line interfaces by exploiting the power of the human visual/motor system. Then in the mid-1980s, advances in computer graphics hardware prompted research on visualization, the use of interactive, visual representations of data to help speed and amplify human understanding. The early focus of visualization research was on individual analysts trying to find stories with data, first in the area of scientific data and then more generally with abstract information.

EF: That makes sense. I, as an analyst, often work by myself trying to find what the data is saying. If that was the early focus, what’s changed now?

JM: Since 2000, the research focus has expanded. Now we’re going from the visualization of an individual analyst to visual analysis – in other words, the use of visualization in larger processes of “sensemaking”. Reduced to its essence, visual analysis has a four-part cycle. First is the focus on a data-oriented task. Second is the need to forage for relevant data. Third, you visualize the data. And then you perform an appropriate action in response. Given a task, analysts forage for relevant data, which is mapped to visualizations that exploit the power of the human visual system. Visualizations lead to findings, which prompt actions. When the actions are new data analysis tasks, the cycle repeats. There are also internal cycles in this problem-solving process. For example, visualizations can indicate the need to forage for new data.

EF: I see that. Before I ever used Tableau, I would get a file of some data, do some analysis and then realize I needed more or different data to complete my analysis. It was frustrating.

JM: Yes. Now you see why commercial adoption of visual analysis depends critically on easy and efficient connection of visualization technology to databases. Thus we’ve linked these two success stories of computing. Chris Stolte’s Polaris system from Stanford (which was described in an earlier CACM research highlight) made a lot of progress on this topic. It formally specifies data views that compile into database queries.

EF: And of course Polaris was the origin of VizQL, which is at the heart of Tableau. But how does linking databases with GUIs relate to collaborative visualization?

JM: Well, my thesis is that the speed of adoption of visual analysis will be determined by a second research topic: collaborative visualization, in which telling stories with data plays a central role. Extending visual analysis from an individual to a collaborative activity increases the visibility and value of the technique. Most analysts must collaborate with colleagues and managers before actions are approved. Data views make data understandable, which encourages collaboration with people who are not skilled analysts. Interactive data views allow people to do their own analysis with data views authored by others.

EF: Yes, pretty much every analysis has some audience or set of participants.

JM: The idea of collaborative visualization is the subject of the research highlight, "Voyagers and Voyeurs: Supporting Asynchronous Collaborative Visualization" by Jeffery Heer, Fernanda B. Viégas, and Martin Wattenberg. The authors describe a prototype web application that includes several techniques for supporting collaborative visualization, and report on user studies involving the prototype. The most interesting aspect of the prototype is a bookmarking mechanism that supports doubly-linked discussions.

EF: Explain that please?

JM: Data views have the property that the same view can be specified in multiple ways. In this paper, the authors describe how to associate a bookmark with a data view rather than the various specifications of the data view, which supports asynchronous discussions about views. The most interesting aspect of the user studies was that their subjects switched between data-driven exploration and social navigation – that is, between being data voyagers and data voyeurs.

EF: I like that distinction: voyagers and voyeurs. As a business person, you find yourself in either role often.

JM: The "Voyagers and Voyeurs" paper represents an important early step in research on collaborative visualization. The authors made the excellent choice to focus their prototype on the US census data set. By focusing on a single public domain data set, they reduced their prototype’s data foraging complexity, thus encouraging their users to focus on collaborative activities. Furthermore, many people are interested in census data.

EF: So if this paper is an important early step, what’s next?

JM: The next step for research on collaborative visualization is to address topics that arise in more fully featured visual analysis applications. For example, most visual analysis tasks involve multiple data sources. Unlike the US census data set, many data sources have security issues. That can make collaboration more difficult. And, some data sources also change rapidly, making asynchronous conversations more complex. Finally, tasks involving multiple data sources often require conversations that compare and contrast multiple data views that must be viewed simultaneously.

EF: Collaboration clearly adds a lot of complexity to any analysis. But it’s critical – everyone works with someone.

JM: Indeed. And by using collaborative visualization, people can work more quickly and appropriately together. Finding and telling stories with data can help people understand the world more clearly. For example, the mortgage-backed security crisis might have been averted if mortgage data had been available for storytelling. The key is to have visual analysis technology and use it appropriately.

Suggested reading:
S. K. Card, J. D. Mackinlay, and B. Shneiderman. Readings in Information Visualization: Using Vision to Think. Morgan Kauffman, 1999.
D. M. Russell, M. J. Stefik, P. Pirolli, and S. K. Card. The Cost Structure of Sensemaking. In Proc. of ACM CHI’93, 269-276.
C. Stolte, D. Tang, and P. Hanrahan. Polaris: A System for Query, Analysis and Visualization of Multi-dimensional Databases. Communications of the ACM 51(11), 75-84, 2008.