Deep Learning models often exhibit undue confidence when encountering out-of-distribution (OOD) inputs, misclassifying with high confidence. The ideal outcome, in these cases, would be an "I do not know" verdict. We enhance the trustworthiness of our models through selective classification, allowing the model to abstain from making predictions when facing uncertainty. Rather than a singular prediction, the model offers a prediction distribution, enabling users to gauge the model’s trustworthiness and determine the need for human intervention. We assess uncertainty in two baseline models: a Convolutional Neural Network (CNN) and a Vision Transformer (ViT). By leveraging these uncertainty values, we minimize errors by refraining from predictions during high uncertainty. Additionally, we evaluate these models across various distributed architectures, including new AI architectures, Cerebras CS-2, and SambaNova SN30.
© The Authors, 2023.
Author Posting © The Authors, 2023. This article is posted here by permission of the ACM for personal use, not for redistribution. The definitive version was published in SC-W '23: Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, Pages 391-394, November, 2023. https://doi.org/10.1145/3624062.3624106