Mobile devices and applications have traditionally been seen as data consumers. However, as their computation capabilities are increasing, they are becoming perceived as potential data collectors and providers, and even as hosts for web services. In fact, mobile web services gain in importance when it comes to deliver real-time contextual data, such as current location or real-time heart rate. Some emerging lightweight frameworks to host web services on mobile devices have been developed. They are recognized for their low resource footprint. However, they are barely tested and the potential of utilizing them in real-life settings is not known yet. In this paper, we present an architecture to test web services hosted on mobile devices as well as some preliminary results. We target to evaluate the QoS of these web services such as their response time, availability, throughput, and scalability and to evaluate the overall performance of mobile device host with main focus on the battery consumption.