Georgia Tech's Bionic Woman  
by Andrew Kerr
November 2006
The CEISMC Gazette

Dr. Ayanna Howard went to college on two different coasts before working at the Jet Propulsion Laboratory. There she applied her knowledge of robotics both to military projects and space exploration. Now she's at Georgia Tech, where she intends to start looking into how robots can assist in the home. The following interview was conducted November 6, 2006.

Q - What do you do?

A - I work on robot brains. The way I do that is I give them the ability to learn. I think intelligence is derived from our ability to extract information from our experiences. We're doing it OK, so maybe we can actually have the robots do it, too.

Q - What's a good working definition of "robotics"?

A - There are so many definitions. It's like, "What is an engineer?"

The loose definition is: "A system that must interact with and in a physical envrionment." Mine is: "A learning system that can interact with and in a physical world." Webster's definition is: "An automatic device that performs functions normally ascribed to a human being."

Q - What specific projects are you working on right now?

A - Deploying sensors in Antarctica. We're working with a scientist who's looking at ice shelf stability. In Antarctica the ice is melting, and melting means that the water rises. Water rising causes chaos in our climate. Antarctica and Greenland have about 60 or 70% of the world's freshwater. So if they melt we would become undersea creatures.

So the scientist has some theories about how you can measure that. The problem is that Antarctica is a big place. When you go out as a scientist you put out a couple of sensors and you have to come up with this global theory based on limited data. What I'm trying to do is actually have the robots take the sensor package and distribute it to the sites that the scientist needs.

Q - Seems the scientists would be delighted not to have to go out into that cold and do it themselves!

A - Interesting enough, yes and no. It's just like with astronauts. If you have a choice between staying in your nice, safe shuttle, or going on the surface of the moon or Mars, you go on the surface of the moon or Mars. Even for a scientist it's like that. They still want to be part of the action. A human will actually do something that's more dangerous than they should because of the excitement.

And the thing is, if you let a machine do the processing for you, you may miss something.

Q - I recall, when you spoke to some high school students recently, that this Antarctica research also has some potential applications to Mars exploration...

A - There's places on earth which we call "analogous sites." We call them analogous sites because the same techniques and technologies that are developed for these earth-based sites can be applied to the moon, Mars, and other planets. NASA actually does a lot of earth science, because it has a direct relevance to the outer planets as well. Even though slipping while driving on Mars is different from slipping while driving on Earth, it's still slipping. If you look at some of the previous issues with one of the NASA rovers, there was a lot of slippage. It got stuck because the sand was so loose. If you can detect Antarctic ice, you can apply those same techniques to detect slippage on Mars.

Q - What's the timetable for sending a person to Mars right now?

A - The first person flight, the sortie missions to the moon, I believe are 2018, so if you continue on after that probably another ten years for Mars. So that puts us at about 2028, and I think the last projection I saw was around 2030 for the first person on Mars. All of the international agencies are focused on space exploration right now: European Space Agency, Japan Space Agency, Chinese astronauts as well.

Q - Why do we care about Mars?

A - There's really two aspects. One, if there is life there, it may help us to understand the formation of life here on earth (Mars has so many similarities to Earth, in terms of the theory about how it was formed). The other thing is that if there was life, and now it's not there, what happened to it? Understanding that process of extinction will help us to understand if this is something that we need to be concerned about. Mars over any other planet holds the answers to those questions.

Q - Where did you grow up?

A - Pasadena. I moved there when I was 7 or 8, and stayed there until I was 33. In between, I went to undergrad in Rhode Island, so of course that was four years out of my life. When I was at USC I lived in L.A., so that was 5 or 6 years out of my life.

Q - You went to college in Rhode Island? Were you trying to get as far away from your parents as possible or something?

A - My parents said they would support me through college as long as it was not near home. They wanted to have me exposed to life outside and they saw that a lot of the kids in the area weren't expanding their minds and weren't seeing the world outside of California. They said they'd support me if I went out of the state.

Q - That's a very common accusation made about California. That it's like a whole other planet!

A - [Laughs.] It is! Of course, at the time all my friends were going to in-state and I was going [dejectedly], "I gotta go to out of state..."

Q - Why Brown Univerisy?

One, because they didn't have grades.

Q - A lot of schools do that, now.

A - Now a lot do, yes. MIT does it for the first year. But at Brown, you could go through the whole program and not have a grade ever. I thought that was kind of nice because I was always so worried about grades. So I took music, I took dance, I took these things because I could and I didn't have to worry about the grades. And I think it was a beautiful experience.

Q - Even though there were not grades, for the sake of kids who are reading this it should be explained that you could flunk out of Brown...

A - You could definitely flunk out of Brown.

Engineering was one of the few very structured majors. And the thing about engineering was that you started off freshman year taking engineering courses. You didn't officially declare anything until your sophomore year, but if you were going to do engineering you had better know by semester one because you were already taking courses.

Q - And clearly you did. So what got you interested in engineering in the first place?

A - Originally I wanted to create artificial limbs for people. I didn't even use the term "robotics." It was "bionics." Somebody told me if you want to work on people you have to go to med school. Then I took biology and I hated it. I struggled. And I always liked math and I always liked the hard sciences, so after talking to people I thought, "I don't know if I like this med school stuff! This is not fun for me!" And people said, "Well, engineering! You can actually create the actual arm itself, and then you would work with a doctor who'd connect it." And I was like "That's what I wanted to do!" By my junior year I decided I was going into engineering.

Q - What sort of high school did you go to?

A - I was at a public school where my tennis coach was the math teacher. Football was the reigning thing. It was a sports school, totally a sports school. So, I was in the honors program, maybe 20 kids out of...I think our class was maybe 300. It was a pretty big school, and honors was about one classroom.

I also played tennis and badminton, and I placed and everything. I was in the choir, which was actually a gospel choir. My nickname was, "Oh she's the smart one leave her alone." 'cause on the bus, the bus experience was horrible! You got on the bus, everyone was ragging and stuff, but anytime anyone messed with me they'd be like, "Oh she's the smart one, leave her alone!"

Q - Wow! I sure didn't get a free-pass like that! Though I think it was worse for me in middle school than high school.

A - High school was when the fights started. Now they worry about weapons where I went. Of course our generation was a little gentler. Now there are gang problems in the school and all that, but back then it wasn't the best school--it was a sports school, which just meant that academics wasn't as big.

Q - Were your teachers good? Were they so-so?

A - I thought my teachers were very good. But then I went to Brown and I realized I was at a total disadvantage. In high school I took AP Calculus, AP Physics, and I got it, I understood it. And then, when I took calculus at Brown, I was like, "Is this the same course? It doesn't look like the same stuff that I know!" [Laughs.] But I didn't know that until I went to Brown. I didn't realize that my education had actually been very limited.

Q - Did you feel like a fish out of water at Brown?

A - Coming from a public school, that was an anomaly, even in [Brown's] African-American community. Everyone came from boarding schools or private schools. But it totally prepared me for grad school. My grad school was a breeze because as an undergrad I realized I was in classes I didn't understand. ...I re-taught myself calculus, going to the libraries just so that I could keep up. I learned a lot about how you do self-study. So the experience at Brown really laid the foundation for academic success because it was so hard. And the other thing is that a lot of the students who started off with me in Engineering dropped out of engineering, even though they did come from boarding schools and private schools.

Q - So then you returned to Cali, to grad school at USC...

A - And also, JPL [Jet Propulsion Laboratory]. I started working in the summers at NASA's JPL after my freshman year. I applied for a fellowship and JPL was my sponsoring company, and they said you can work while you're going to school. Plus, I missed California. The only grad schools I applied to were in California.

Q - You just can't beat the California weather.

A - You can't! I was ready to come back to a warm climate!

Q - You were doing robotics as an undergrad, and now you were at JPL. Something must have changed.

When I went to grad school I was controls and manipulators. That's what I said on my graduate application, "I want to do controls and manipulation." But when I went to JPL they didn't put me in the robotics group. They put me in the non-linear control group. It was a neural networks group; a learning group. That exposed me to hardcore algorithms, hardcore programming, hardcore neural networks and learning, and I liked it!

That's when I learned vision and took a vision course. I was doing vision at JPL because we were processing these flying objects in the air - we were trying to evaluate how these things looked and say "friend or foe?"

And so I would say JPL really changed the focus of what I was doing in my thesis and where my interest was.

Q - What did you work on specifically? Or is it all top-secret stuff?

A - Preventative systems.

Q - A euphemistic "preventative systems" or preventative systems?

A - Preventative systems. What they wanted to do is ...it's kind of like when football players and coaches put these playbooks together. The military actually has playbooks. You can go to Iraq and they have a playbook, and Korea has a playbook. So the theory was that if you can identify some of the elements that are in the field, you can take that and possibly map it to the playbook. So if I see three [aircraft] in a certain formation, this is probably their attack strategy.

The next stage is how to identify what those icons are in the field. If you're looking at the terrain and you see something flying - one, can you find it as it flies by; two, can you tell me what it is, what class it belongs to; and three, can you tell me where it's going? That required a lot of vision processing with neural networks.

Q - When did you start at JPL?

A - I started working in that group in January '94. I liked the money. In grad school you're poor, and I was tired of being poor! So I started off around 20 hours. By the time I was doing my dissertation I was full-time--which extended my duration in grad school. So that's the negative. I could have done my PhD in three and a half years and it took me four and a half.

Q - Why did you leave JPL for Georgia Tech?

A - The vision for space exploration came in 2004, and NASA's response to that was first of all a total restructuring. The reason I started working at NASA was because of the excitement of the research. Now if you take that away, either I keep on doing the research somewhere else, or I go make a lot more money. And by that time I had an MBA, so, a whole lot more, at least twice! So the opportunity came, and I said, "OK I'll look at academia, because I really think I still want to do research."

Q - Is it weird having all this attention? Being featured in Time magazine and an upcoming proposed PBS special?

A - It was weird then...I like what I do. I think it's exciting. I think anyone who does what I do must have the coolest job in the world. Is that interesting to other people? Not quite sure if it is.

Q - It's obvious of course that we're always looking for role-models. Are there a lot of African-American women in engineering?

A - African-American? No. Women? No. [Laughs.]

Q - What's on the horizon right now?

A - I'm interested in looking into more socially-assistive robotics. I would like to expand my research to focus on robotics for the home, as well as for space. How do you have interactions so that I can teach a robot to go get me my cup of coffee--me being an elderly grandmother--how can I do that versus me as an engineer who programmed it?

For example, if I know how to pour water into a cup...So I have a cup, and I'm pouring it, and I teach a robot how to pour. The difference is that, say you're left handed and I'm right handed. This action [pouring with the left hand] is the same as this one [pouring with the right hand], but in the robotic world those can look like two different actions.

I tell my students, don't promise a system that can learn everything. I [as a robot] should be able to tell you what it is that I can learn, and if I don't know it or can't learn it I should be able to know that and be able to tell you. I should be able to say "that's too hard," just like a kid.

Q - Is it safe to assume that the elderly will be the first beneficiaries of these robotics in the home?

A - I think the elderly is the easiest thing to attack, because there's just an obvious need. When we can get health insurance and Medicare to actually agree that a robot is a medical device, of course that will also open up a lot more doors. Today, if you buy a wheelchair, it qualifies, but if you buy a robotic wheelchair, it's not qualified.

Q - In an article I wrote last month I made a joke about how the world of the future hasn't really arrived as it was promised decades ago. "Where are our robot servers?" I said. I thought I was being cute. What do you think?

A - So there's what I would call robotics in everyone's home, and then there are classical robots (like the robotic server). I think both of them are already a reality, but soon robotics in everyone's home will be synonymous with computers in everyone's homes. Everyone has a computer. If you have a cell phone, you have a computer. Now back in '89 that wasn't a reality, computers, that was a big thing. Cell phones? You had pagers. And [the perception at the time was] if you had a pager you were either a doctor or a drug dealer--

Q - --and none of the kids in my high school were doctors.

A - [Laughs.] But even I had a pager (my parents didn't know). But nowadays people don't even think about it. It's going to be these autonomous systems that are going to do functions that we would call robotic, something as simple as your washing machine that, when you come to it, you'll actually say "open" and it will open and a tray may come out and you put the laundry in, and you communicate. People will say, this is a washing machine, and they probably won't even say it's an intelligent washing machine, just a washing machine. Research from robotics will find itself in everyday items.

And then the classical--Rosie the Robot (from the Jetsons), or an R2D2--I think those are also going to be in existence. I think the cost point is still going to be high enough in the next 10 to 20 years that it will be more of a novelty than a part of life. But the robot server, you can actually buy one now. You may not want to pay for it, but it exists. But it won't be at a price point where you can go to Costo to buy it, that's the difference.

Robotic vacuum cleaners have been in existence for a while before the Roomba came out. What the Roomba did was figure out how do you get the technology to a price point where people will pay for it.

Q - You have those cute robot dogs in your lab. What are you doing with those?

A - We want to send these astronauts to the moon, to Mars, and there isn't a way to determine who should do what--the robot or the astronaut. Say you want to build a habitat. You've got to search for the site, you've got to clear the site, you've got to put in the poles, and so on. What I was trying to do was come up with a theory about how do you determine this task allocation scheme between humans and robots based on cognitive studies (I didn't put the excitement value in).

If it's in the morning, and you're ready to go out, your performance as a human is going to be much better than if you've been doing it for eight hours, whereas a robot's performance is pretty much going to be the same. The problem with the robot is that their variation in performance will depend on the difficulty of the task and the complexity of the environment, so if it's a very difficult task their performance may not be so good.

So it's taking all those factors - cognitive work load, environmental complexity, and task complexity, and coming up with a system that comes up with a configuration that says this is what the astronaut should do, this is what the robot should do.

With the robot dogs, that was how we tested the theory.

Q - Walk me through the dog experiment.

A - The only scenario I tested was habitat construction. I had these blocks that were colored different colors. So, the task was: look for a habitat block; go to the block; and then move it to some other location. Those were the steps of habitat construction. So the dog would be out there, look around, see the block, go to the block, get to the block, and push it. Not entirely true habitat construction, but it allowed me to see if the theory could work with a robot platform. The interface with the human actually had a 3-D environment of the terrain with the blocks, and then a visual camera, so a human could see what the robot could see. The human had a control interface that could control the robot - up, down, left, right which is how we monitored the human performance.

Q - Regarding science-fiction, do those films, like Asimov's I Robot and stuff like that--do they have any relevance to the reality of robotics in 2006?

A - All of them have aspects. For example I Robot...One of the big things was that he had this emotion chip. So there actually are robotics people that are working on emotions and giving robots the concept of emotions, either understanding emotions or themselves exhibiting emotions.

Q - Why on earth would you want a robot that exhibits emotions?

A - OK...

Q - [Laughs.]

A - I don't belong to that camp, but there's one reason that I've found of value, the others I'm not sure. Some of our emotions deal with survivability. If I have fear, I go away from [the source of that fear]. So if you gave a robot fear of obstacles, if you see a rock you'll stay away from it. The concept of avoidance is based on fear.

Humans have desires. If you give a robot the concept of desire you could say, you want to go to the goal. So you say the goal is your desire. As the robot gets closer it feels, "Yeah, I'm getting happier and happier and happier!" So that drives it.

The other thing I've seen which is probably of value...When you're dealing with human/robot interaction, maybe you could design robots so that they recognize certain things. i.e., If a person is doing something, and they get frustrated, anyone nearby could look and know "Oh, she's not having a good day." If it was a robot, maybe that's a cue of: "Astronaut needs help." But again most people don't focus on that aspect.

At MIT, there is a lot of research on the emotion stuff. One way they recognize emotions is based on voice-influx. Typically if you're happy you'll have a higher tone. If you're reprimanding there's a certain syntax or there's a certain melody in your voice. They're actually able to extract certain voice signatures to say this person is mad at me, and then this person is happy, things like that.

Q - Does it work across languages?

A - [Dr. Cynthia] Breazeal, and others, have shown that it does. Again it's not necessarily about recognizing the words, it's about recognizing the tones and influx.

Q - What about the future of robotics as a career?

A - Ten years down the road, robotics jobs will be a definite call. It'll be an industry. I'm now starting to see positions that specifically say "robotics," whereas ten years ago if they were doing robotics it would say "Wanted: electrical engineer that knows hardware; Will: work on robotics."