“There was never a time you could get the majority of people in Alabama or Mississippi, or even southern Delaware, to vote to end segregation. What changed things was the rule of law, the courts. Brown v. Board of Education was ushered in by a movement, but it was a legal decision.”