The recent surge in computer vision applications has caused visual privacy concerns to people who are either users or exposed to an underlying surveillance system. To preserve their privacy, image obfuscation lays out a strong road through which the usability of images can also be maintained without revealing any visual private information. However, prior solutions are susceptible to reconstruction attacks or produce non-trainable images even by leveraging the obfuscation ways. This paper proposes a novel bit-planes-based image obfuscation scheme, called Bimof, to protect the visual privacy of the user in the images that are input into a recognition-based system. By incorporating the chaotic system for non-invertible noise with matrix decomposition, Bimof offers strong security and usability for creating a secure image database. In Bimof, it is hard for an adversary to recover the original image, withstanding a malicious server. We conduct experiments on two standard activity recognition datasets, UCF101 and HMDB51, to validate the effectiveness and usability of our scheme. We provide a rigorous quantitative security analysis through pixel frequency attacks and differential analysis to support our findings.