Demand response is an important tool for utilities to manage load during peak periods. While the effects of demand response programs on peak load reductions are well studied and intuitive, assessments typically fail to recognize the potential for off-peak behavioral responses that may mitigate the total benefits of the program. Using smart meter consumption data on residential air conditioning units enrolled in a direct load control program, this paper examines changes in consumption prior to and after curtailment events. The results suggest substantial increases in off-peak consumption, which reduce energy, monetary, and environmental benefits of the program by over 40%.