“For instance, everyone's used to the idea of convolution now in image space. But if you think about it. And I've had people tell me who work on convolutional neural networks that they don't build any structure into their system. It's just a neural network. But of course, as soon as you build the compositional structure into a neural network, you are taking a position on some regularities that are in the input signal and so on. And you're taking advantage of that so that you don't have to learn a whole fully connected network, but you just learn some compositional kernels.”