“It can be safely asserted that since early Colonial times, the North has had a distinct race problem. Every one of these States had slaves, and at the beginning of Washington's Administration, there were 40,000 black slaves and 17,000 black freemen in this section.”