“When you're allowing the Executive Branch to deprive somebody of a constitutional liberty without any process, that is something that affects all Americans because that's a precedent that can be used.”