Internet of Things (IoT)-based smart factories offer the manufacturing sectors a great opportunity to embrace the fourth industrial revolution (Industry 4.0). The real-time monitoring of manufacturing operations in an Industry 4.0 needs to be ensured by the deployed technologies, like Artificial Intelligence (AI) and Big Data analytics. The overall purpose is to improve the outcomes of the production process. However, Industry 4.0 becomes vulnerable to different potential attacks as the communication takes place via public environments. In this article, an authentication and key agreement method has been suggested to secure the communication that can occur in a Fog-based Industry 4.0 environment. The security proposal provides secure mutual authentication along with key establishment between various smart industrial devices and fog servers, as well as between fog servers and cloud servers. The security analysis and comparative study reveal that the proposed method can mitigate various potential attacks, and it also offers important security and functionality attributes as compared to those for other competing schemes.