“Frankly, I had thought that at the time Roe was decided, there was concern about population growth and particularly growth in populations that we don't want to have too many of.”