Evidentiary rules for criminal trials disallow various forms of probative evidence. Conventional wisdom assumes these rules benefit all defendants, whether guilty or innocent, and thus reduce wrongful convictions at the price of more wrongful acquittals. We show the conventional view only holds under stylized conditions. We further identify properties of evidence generation mechanisms under which the conventional view is backward: an evidentiary rule will harm the innocent and protect the guilty. However, if adjudicators place too much weight on the evidence, its exclusion can reduce both wrongful convictions and wrongful acquittals.