© Springer-Verlag Berlin Heidelberg 2016. In ubiquitous environments, it is important to recognize the situation and deliver services accordingly. In addition, it is equally important to have a fast response time. The existing context-aware activity recognition engines have good recognition rates; however, they consume lots of time to produce feasible results. Our focus in this research is to reduce the time required by eliminating the need for ontology matching (in context-aware activity manipulation engine) and extend the rules. In addition, we incorporate the sliding time window concept to retain activities for a longer duration and maintain their relevance using ontological data for a better accuracy. The proposed scheme has increased the overall accuracy against the existing system by 12.6 % for individual activities relevance and 6 % for high level activities.