Visualizing Data Part One: Looking vs. Seeing 
October 2008

Ever since Ogg documented the success of a mammoth hunt on a cave wall, humans have recognized the value of visualizing data. John T. Stasko, Professor and Associate Chair at the College of Computing & GVU Center at Georgia Tech, explores ways that we can speed up the processing of information.

John and I began our conversation talking about koi fish, and then moved on to the way we look at the world in general. The next parts of this interview will take a look at the specific projects he has been working on, including new ways of fighting the "war on terror" and solving crimes.

Q - How is your koi pond coming along?

My wife just called me earlier. I think we have a couple of baby koi. We used to get a number of babies every year, but it's tougher for them to survive now. But it looks like one or two may have survived.

Q - Does backyard wildlife eat the baby koi?

I think the koi may. Snakes will also go into the water and stuff. We have to sometimes remove the snakes from the net, which is no fun because they're usually alive and they're entwined. And then baby koi sometimes just get sucked up in the filter!

Q - We humans have been visualizing data for a long time. Even before there was written language there were pictures. What is the psychological background behind what you do? I guess we might start with your information canvas, with which you did a study showing that in some instances people retained information better visually than textually.

One of the angles we take is that imagery is able to be accessed much more in parallel. I can take in a lot of data. I'm looking all over the picture. My eyes can look in only one place, so I'm still doing a sequential scan. But that's more of a random access process as opposed to text, which is inherently sequential. We go word-by-word and we work our way through.

At some level we can think of our visual perceptual systems as being essentially a wider pipe onto the information pipeline. We can take in more per unit time. If you look at kind of information bits per second, it's the winner. With text I've got to read over it and so at some level the pipe's a lot smaller. So that study with the info canvas...at some level it was unsurprising because you show me this picture of a lot of different stuff, I'm able to look, look, look, and if I have a reasonable memory I can digest that, as opposed to you wrote a text description of all that stuff and I had to work on it. I think people are very good at visual encodings because as long as we're awake we're getting practice.

But the interesting thing in that study was the visuals. It wasn't just somebody remembering there was a red beach ball and a plane in the sky and a crab on the beach. There was a mapping to that. The plane was high in the sky which meant the stock market was up a lot that day. The beach ball was red, meaning that traffic was horrible on I-75 going home. And people were even able to do that second level of the mapping better and faster when compared to reading it in the text. We talk about it from a perception point of view about being highly parallel; I can take in a lot of different things concurrently.

One of the examples I often use for how good we are at perception is if you're out house shopping and you either walk into one of those new subdivisions being built, and you walk in the bathroom, and if the wallpaper is mislaid, the two sheets by like a millimeter, when you walk into that room your eye goes right to it. We're just these incredible visual-processing systems.

Q - It seems that one of the most critical skills we have is our ability to quickly detect breaks in patterns, whether it's mislaid wallpaper or a person in a crowd who seems to be acting a little bit "off."

We do grouping very well. We see what's together, what's apart, etc. We do similarity/difference. We look for outliers. We have the visual properties of size, color, etc. We also talk in my info viz (visualization) course I teach here about pre-attentive processing that human beings do. We do some visual analysis pre-attentively, kind of pre-thinkingly.

A good example is you put up a visual scene for a very short period of time, like a quarter of a second, or a half a second. And you ask someone a question about it and determine if they can answer it correctly.

[Please take a moment to watch this video before continuing.]



Two different properties we may look at might be color versus shape. I put up in that half second a scene. I have a whole bunch of blue circles and I put a red one in. What if I only show that for like a tenth of a second or a quarter of a second, and you ask someone was there a red one there? They'll tell you there was, because color is so amazingly pre-attentive. We just pick it up.

But if you use shape, if they're all red circles, and now you put in a red rectangle or a red square, we can do it pretty well. But as that time delta shrinks it gets harder and harder.

Then you can do interesting combinations. You have squares and circles and you do red and blue, and say was there is a red rectangle or something. The composite of the two blows it away. It's no longer pre-attentive. You basically drop into a serial search procedure. And so anything a quarter second, half a second, you can't do it.

The implication is that if you're doing certain analytical tasks which are really important in your application, in your system, whatever you're doing, and you make some search, we'll use a strongly visual pre-attentive property to encode a search item. We see a lot of bad user interface design design and bad visualization design when they're using the wrong visual property; they're using length of something when they should have done whatever. But again it goes back to what's your ultimate task that you're trying to use the visual encoding for?

Q - Are things trending more throughout society at large towards using more visualization? I think of the evolution of the computer desktop, for example.

I think so in general. It wasn't that long ago, 25, 30 years ago, that our machines didn't do graphics all that well. Now we've had the desktop metaphor and GUI [graphical user interfaces] interfaces since around '80. They're still being refined a bit, it's still fairly new, and we're still feeling that out.

I think using visualizations as analytic aids the way I do as the data-sets get bigger...that's a newer area, and I really see that blossoming. I think that we're getting better tools. Before there might have been one company that had something and it was very expensive. Now you can download some open-source free thing to get that.

It's not a panacea though. I worked in an area, back in the late 80s, early 90s, when I was coming out of grad school, visual languages, visual programming. It was a big thing to say back then that "We're going to do computer programming. Rather than having textual languages like Java or C++, we're going to have icons and pictures! And we're going to string them together and have arrows and connect them!" That was a very hot research topic. And it bubbled and it never worked, it never hit.

It's because text is just better. You get the syntax, it's specific, it's detailed, it doesn't use up the real estate that imagery and pictures do. So that kind of general visual language, visual programming idea has sort of fallen by the wayside. There are some specific domains where interactive visual programming tools are very powerful, commercial tools, and they're used a lot, but as a general replacement for Java or C++ or Fortran or whatever, no, not going to happen. Text is very efficient and very appropriate for that.

Coming soon: A look at how visualizing information can solve crimes and save lives.