Total Productive Maintenance (TPM) is a critical activity that significantly reduces lead times and uncertainty in Make-To-Stock (MTS) production systems, thereby increasing the efficiency and profit margins of the associated firm. TPM problems can be set up as semi-Markov decision processes (SMDPs) and solved optimally using classical dynamic programming (DP) on small-scale problems. However, on large industrial-scale problems, DP breaks down, and one must then resort to an artificial intelligence (AI) technique called reinforcement learning (RL). This work presents a new AI algorithm for solving SMDPs, called iSMART, an acronym for imaging Semi-Markov Average Reward Technique. iSMART requires a significantly lower modelling and computational effort than classical DP, where estimating the transition probabilities can itself be very time-consuming and mathematically challenging for large-scale problems. Further, unlike previous RL algorithms for SMDPs, iSMART does not need exploration decay. This means iSMART eliminates an additional parameter that requires significant tuning in the traditional RL-based solution approach. Modern AI based in deep learning seeks to reduce dependence on tuning parameters. iSMART is designed in this spirit for solving TPM problems in MTS production systems, where it is shown to deliver optimal solutions on small-scale problems and near-optimal ones on large-scale problems.