“The perception of linked fate and that feeling of being always on the spot as a representative of the race, at least in mixed company, are features of African American life that predate affirmative action and arise outside of its presence.”