Interest in and ownership of smart home voice assistants like Amazon Alexa and Google Home devices have exponentially increased in recent years. Many people may purchase or be gifted such devices without knowing their potential for connecting with other home technology, listening to private conversations, sharing information with companies, and creating problems due to misunderstanding vocal commands or technological capabilities. Concerns and worries about these devices may be exacerbated over time or by a specific incident. To understand reactions to such situations, we conducted semi-structured in-depth interviews with ten people who reported different types of worrying incidents with a range of smart home devices and their reactions to reduce that worry. The two dominate reactions were restricted acceptance or discontinuance of the smart home technology, while three other interviews revealed nuanced reactions on the acceptance-rejection continuum.