Interest in and ownership of smart home voice assistants like Amazon Alexa and Google Home devices have exponentially increased in recent years. Many people may purchase or be gifted such devices without knowing their potential for connecting with other home technology, listening to private conversations, sharing information with companies, and creating problems due to misunderstanding vocal commands or technological capabilities. Concerns and worries about these devices may be exacerbated over time or by a specific incident. To understand reactions to such situations, we conducted semi-structured in-depth interviews with 10 people who reported different types of worrying incidents with a range of smart home devices and their reactions to reduce that worry. Conducting a thematic coding analysis, we detail how each case study shows a person's worries about their smart home technology developed vis-à-vis the incident or over time, and their strategies to alleviate their worry. The two dominant reactions were restricted acceptance or discontinuance of the smart home technology, while three other interviews revealed nuanced reactions on the acceptance-rejection continuum. For each interviewee, we highlight their technology use, any major incidents, and their psychological processes leading up to their actions to reduce worry. This provides an in-depth look at worry around smart home technology products themselves, not their ability to perform, and how discontinuance, restricted acceptance, and other reactions reduce those worries.