Network Intrusion Detection systems (NIDS) are essential for organizations to ensure the safety and security of their communications and information networks. Signature-based IDS has good detection capabilities for known attacks, with fewer false alarms, however, it is not effective against Zero-Day or unknown attacks i.e., it has low recall (high false negative rate). In contrast, anomaly-based IDS focuses on deviations of the traffic pattern and uses those deviations to evaluate incoming traffic and determine the chance of anomaly, even when faced with unknown attacks. Using Reinforcement Learning for intrusion detection gives the ability of self-updating the model while detecting the incoming attacks, to reflect new types of network traffic behavior. The use of Multi-Armed Bandit approaches for hyper-parameter optimization in unsupervised anomaly detection problem in Internet of Things (IoT)-based smart infrastructure has gained some interest in the research community. The method achieves better detection accuracy by applying a novel probabilistic cluster-based reward mechanism to non-stationary multi-Armed bandit reinforcement learning. This approach works by optimizing the set of hyperparameters of the underlying unsupervised anomaly classifier based on the cluster silhouette scores of its outputs. This paper explores improvements in the existing works leveraging multi-Armed bandit techniques for unsupervised anomaly detection in smart homes for optimized intrusion detection. We evaluate notable multi-Armed bandit algorithms such as non-stationary UCB1 and EXP3 algorithms on network traffic and compare their performance with adversarial non-stochastic contextual bandit EXP4 algorithm. We observe that we achieve significant improvement in IDS accuracy and performance. This work can benefit the future research in this area with different smart environments and different attack scenarios.