Recently, the concept of Internet of Vehicles (IoVs) conquered the automotive industry, academia, research fields, vehicle manufacturers, etc., where vehicles are 'intelligent ones' capable of providing a wide variety of applications, such as traveller/driver safety, infotainment, traffic efficiency, reduced congestion, less pollution, etc. Ensuring proper authentication and secure communication are the major challenges of an IoV scenario. However, only limited works are available for authentication and communication, among them the 'lightweight property' is missing. Hence, in this paper, we design a lightweight mutual authentication protocol in an IoV scenario using cryptographic operations. The proposed protocol also enables a device and a server to establish a secret key, which can be used for secure communication, while minimizing the computational cost associated with the process. The protocol is implemented on two types of communication models, such as two Raspberry Pi's connected via an intermediate desktop computer acting as the Trusted Authority (TA) and two Raspberry Pi's connected via the cloud (here, Vehicle Server). The performance analysis results based on computation and communication cost show that the proposed protocol performs better than existing systems.