Though clearly distinct in nature and procedure, both regulatory agencies and courts frequently rely on similar instruments to sanction the same or very similar kinds of illegal behavior. In this article, we develop a theory of the use of criminal sanctions in addition to regulatory penalties. We show that, even though it is generally more effective to have a penalty imposed by a regulatory agency rather than by the courts, under some conditions it is optimal to have both. The article provides three arguments: agency costs when delegating law enforcement, legal error, and collusion between a regulatory agency and an offender. The objective of the article, though, is not limited to the determination of the theoretical conditions that can make the use of both sanctioning schemes optimal. Our analysis is also relevant to the application of a specific legal doctrine, the Double Jeopardy Clause.