“When a man has emerged from slavery, there must be some stage in the progress of his elevation when he takes the rank of mere citizen and ceases to be the special favorite of the laws.”