“The only people to whom myself and the immigration issue is toxic are to the well-heeled committed Remain voters, the sort of people who live in the Hannan and Carswell world.”