“All rights, however noble, impose duties on others, even if in many cases we wholeheartedly agree that the benefits outweigh the costs. And the trade-offs can be tricky.”