Some bioethicists argue that conscientious objectors in health care should have to justify themselves, just as objectors in the military do. They should have to provide reasons that explain why they should be exempt from offering the services that they find offensive. There are two versions of this view in the literature, each giving different standards of justification. We show these views are each either too permissive (i.e. would result in problematic exemptions based on conscience) or too restrictive (i.e. would produce problematic denials of exemption). We then develop a middle ground position that we believe better combines respect for the conscience of healthcare professionals with concern for the duties that they owe to patients. Our claim, in short, is that insofar as objectors should have to justify themselves, they should have to do it according to the standard that we defend rather than according to the standards that others have developed.