The advent of Internet of Things and artificial intelligence in the era of Industry 4.0 has transformed decision-making within production systems. In particular, many decisions that previously required significant human activity are now made automatically with minimal human intervention via so-called digital twins (DTs). In the context of maintenance and reliability modeling, this naturally calls for new paradigms that can be seamlessly integrated within DTs for decision-making. The input data for time to failure needed in reliability computations are directly collected from the work center in a digital setting and often do not satisfy a known distribution. A neural network (NN) is proposed here to bypass this difficulty within the DT. Further, an algorithm inspired from machine learning is employed to solve the underlying semi-Markov decision process, whose transition model is captured via the NN. Numerical studies are carried out to demonstrate the usefulness of the approach. Finally, convergence properties of the algorithm are analyzed mathematically.