Leslie P. Kaelbling

“Fundamentally, the way we think about it is that we decompose the computation that's in the robot's head now into two parts. The first part is in charge of taking the sequence, the history of actions and observations, and trying to synthesize them into some representation of a belief for a probability distribution about the way the world might be and then another module that takes that belief and decides how to behave.”

“And so any approach that works effectively in a domain like this is going to have to handle very large spaces, very long horizons, and really lots of uncertainty. So we have kind of a standard structural decomposition to this problem. We call this belief space hierarchical planning in the now. I'll decode what that means a little bit.”

“So the dimensionality of the space is kind of unthinkably high. It's also not exactly clear what constitutes an object here. If you were going to behave in this world, it would be a very long sequence of primitive actions that you would take in order to clean this kitchen. And also there's just a fundamental amount of uncertainty in this problem, right? So you don't know what's in the blue bowl or what will happen if you try to pull out a certain thing. You don't know when the people are coming home or what they want for dinner all sorts of stuff you don't know.”

“And just again, to kind of give you the motivation. I really want a robot. This isn't my kitchen, by the way, just in case you were worried, my kitchen. But imagine that you had to clean this kitchen or make breakfast in it or something. It would be very hard. And imagine programming a robot to do it. It's extremely hard. And so one thing that's useful to do is to think about what makes this problem hard. So one of the things that makes it hard is that there are lots of objects.”

“So one set of possible kind of general ideas includes convolution in space also and time. Maybe understanding the kinematics of the system that it's connected together in joints and segments, a notion of planning to move through space, being able to do causal reasoning-- if I were to do this, what would happen? -- Abstracting over individual objects, various kinds of state and temporal abstraction and so on. So our view-- I don't want to commit to a particular list-- but is that there's a list of structural principles that are pretty generic and very broadly useful and we should build them in.”

“So just as convolution gives us a great leverage when you apply it to the right part of the problem, then the intuition as well, hopefully there there's a few mechanisms, hopefully not like 100 mechanisms, but maybe 10. And then if we figure out how to use those mechanisms to bias learning and to structure behavior that we could learn robust ways of behaving that are efficient and so on.”

“For instance, everyone's used to the idea of convolution now in image space. But if you think about it. And I've had people tell me who work on convolutional neural networks that they don't build any structure into their system. It's just a neural network. But of course, as soon as you build the compositional structure into a neural network, you are taking a position on some regularities that are in the input signal and so on. And you're taking advantage of that so that you don't have to learn a whole fully connected network, but you just learn some compositional kernels.”

“So I'm still there, OK? So I'm still in I'm still trying to figure out how we can design an architecture that can learn efficiently. And so the research strategy that I have really adopted, I work closely with a colleague, Tomás Lozano-Pérez. Our strategy has been the following, which is to try to think of some very generic representation and inference mechanisms and build those in and then figure out how to learn the rest of the stuff. And we're all used to I think by now the idea of some representation that inference mechanisms that we would want to build in.”

“So I'll just tell you some story because people usually like stories, and it's kind of the afternoon. So and this is related to the question about reinforcement learning, probably, right? So how did I get into this whole thing? When I just finished my undergraduate degree, which actually was in philosophy, weirdly enough, I went to work at a research institute while I was starting my PhD. And they had this robot nobody really knew actually very much about robotics. So And it was my job as the brand new person to try to get the robot to drive down the hallway.”

“So all these constraints need to somehow come together into a way of building intelligence systems. Actually, I would stop here for a minute just because it's a convenient spot and see if there are questions. I see some red Q&A button. So maybe someone can ask.”

“So if humans have to engineer a very complicated system, then it has to be the engineering process has to have some modularity to it because humans are really bad at understanding one big messy system. They're good at understanding pieces and parts that work together. So it may be that we have to take a modular design approach in our engineering efforts for intelligence, not because the intelligence needs to have that architecture, but because we, the human engineers, need those tools for actually building a system.”

“I also know just some physics and variance about the worlds that my robot's going to operate in. And maybe humans don't have this built in explicitly, but they almost surely have a built in implicitly. And I also have some other constraints as an engineer who's trying to make intelligent robots, which is that humans are the engineers, right?”

“And so then the question is can I somehow take pieces and parts of all these ideas, some human programming, some robot learning in the wild, some kind of search or evolution offline, some inspiration from humans. Can I take all those things and put them together and see if I can find a way to engineer intelligent robots? So that's basically what I'm up to.”

“So there are a bunch of ways you can think about the problem. I mean, one would be to say, "Oh, I'm really lazy. I don't really want to think very much about working in the factory. It seems awfully hard. I will just make a robot that has roughly an empty head. It doesn't really know very much at all. And then it just has to interact in the world and learn everything by interacting." But of course, you don't really want a robot that comes to your kitchen and begins to learn about physics, right? That would break a lot of dishes.”

“But the problem is now I've written down an objective function. I've said, "Oh, if you could tell me a distribution over possible worlds that you'd like this program to work well in, then I know in a certain mathematical sense with the best program is." But now my problem as the engineer, as the person who is in the robot factory, which is again, the kind of maybe analogous to the problem of nature, is I have to figure out how do I how do I find this program that's going to be good and all these situations?”

“So I imagine that there's some distribution over possible environments that the robot could find itself in when it actually goes out into the world, right? So maybe it's going to go to houses and the houses are all somewhat different. And once I put that program in the house, maybe it's going to do some estimation or learning. It's going to adapt to the circumstances it's in. My job is to find a program that does a good job of adapting in all the environments that might find itself in.”

“And so what we want to do is think about first of all, what's the best-- what would be the best pi to put inside the robot. How can we think about that? And then we have to think about the problem of how is it that we, in my case, as me, as an engineer, I'm going to find that pi that I should put in my robot.”

“The reason I want to start by backing all the way up to this like very basic control theory picture is that right now there's an enormous amount of argument about how one should make robots. Should they do planning and reasoning? Should they do reinforcement learning? How should we do it? So there's a huge kind of crisis almost in the field about what the best methods are. And what I want to start out this talk by doing is actually thinking about how we can answer that question in a way that's not political or religious, but technical.”

“So the way I think about the problems-- this is kind of a definitely a computer scientist way to think about the problem-- is to think about the robot as a transducer, as some kind of a system that's connected up to the world. And it makes observations of the world. And it takes actions that change the state of the world. And presumably, there's some objective, right? We want to take actions that change the state of the world in some way that we think will be good.”

“OK, so what's my research goal? I come from the machines end of this world, roughly. And what I really want to do is figure out how it is that we can make intelligent robots. And I do this mostly because I'm interested in intelligence more than I'm interested actually in robots. But I think that trying to make a physical agent who goes out and interacts in the world is a really good test bed for understanding what kinds of reasoning and perception and control we need in order to make it an intelligent system.”