“As we leave the E.U., freedom of movement falls away, because it's an E.U. rule... What we then have to say is, 'What then is on the blank piece of paper that is an immigration policy?'”