“When you have incidences like the Trayvon Martin verdict, the erosion of certain fundamental rights like voting, it just reminds us that we're always one Supreme Court justice vote away from losing the progress that has been made.”