Charter schools involve a trading of autonomy for accountability. This accountability comes through two forces—markets through the choices of parents and students, and accountability to government through the writing of contracts that must be renewed for schools to continue to operate. Charter schools are supposed to be more accountable for educational performance than traditional public schools because authorizers have the ability to revoke charter contracts. Here, I focus on one central component of accountability to government: performance accountability or accountability for educational outcomes to charter school authorizers through the revocation or non-renewal of charter contracts. In this paper, I suggest that contract-based accountability for educational performance in charter schools may not be working as proponents argued it would. This article explores some explanations for why there are very few examples of charter schools that have been closed primarily because of failure to demonstrate educational performance or improvement. Future work will need to test if these challenges for authorizers hold in a variety of contexts. The conclusion examines the implications of these findings for the future of charter school accountability.