While Bayesian inference can enhance intelligent probabilistic computing systems, such systems are often computationally expensive and not well suited for implementation on von Neumann architectures. Naive Bayes Nearest Neighbour (NBNN) is a simplified algorithm that performs Bayesian inference. In this paper, we propose a simplified, parallel, and efficient memristive architecture that approximates NBNN. Also, we introduce a simple, and a novel way to incorporate the prior knowledge (probabilities) within the memristive crossbar to enhance classification accuracy with minimal increase in power consumption. We test the algorithm and architecture on the spatial MNIST dataset of handwritten characters. We extend the same architecture to the MSRAction3D video dataset containing spatial, temporal, and depth information, in order to determine how well our architecture scales, as well as to compare accuracy with methods that use machine learning components. Compared to other Bayesian/probabilistic systems, our approach consumes about half of the power due to the use of low power memristive devices. The power numbers are obtained from SPICE hardware simulations. Our architecture can be used in inference applications where speed and low power are of great importance, and a slight loss in accuracy is tolerable.