Skip to main content
Article
Towards High-End Scalability on Biologically-Inspired Computational Models
Parallel Computing: Technology Trends
  • Dario Dematties, University of Buenos Aires
  • George K. Thiruvathukal, Loyola University Chicago
  • Silvio Rizzi, Argonne National Laboratory
  • Alejandro Wainselboim, University of Buenos Aires
  • Bonifacio Silvano Zanutto, University of Buenos Aires
Document Type
Book Chapter
Publication Date
3-1-2020
Pages
497 - 506
Publisher Name
IOS Press Books
Abstract

The interdisciplinary field of neuroscience has made significant progress in recent decades, providing the scientific community in general with a new level of understanding on how the brain works beyond the store-and-fire model found in traditional neural networks. Meanwhile, Machine Learning (ML) based on established models has seen a surge of interest in the High Performance Computing (HPC) community, especially through the use of high-end accelerators, such as Graphical Processing Units(GPUs), including HPC clusters of same. In our work, we are motivated to exploit these high-performance computing developments and understand the scaling challenges for new–biologically inspired–learning models on leadership-class HPC resources. These emerging models feature sparse and random connectivity profiles that map to more loosely-coupled parallel architectures with a large number of CPU cores per node. Contrasted with traditional ML codes, these methods exploit loosely-coupled sparse data structures as opposed to tightly-coupled dense matrix computations, which benefit from SIMD-style parallelism found on GPUs. In this paper we introduce a hybrid Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) parallelization scheme to accelerate and scale our computational model based on the dynamics of cortical tissue. We ran computational tests on a leadership class visualization and analysis cluster at Argonne National Laboratory. We include a study of strong and weak scaling, where we obtained parallel efficiency measures with a minimum above 87% and a maximum above 97% for simulations of our biologically inspired neural network on up to 64 computing nodes running 8 threads each. This study shows promise of the MPI+OpenMP hybrid approach to support flexible and biologically-inspired computational experimental scenarios. In addition, we present the viability in the application of these strategies in high-end leadership computers in the future.

Identifier
ISBN 978-1-64368-070-5
Comments

Author Posting © The Authors, 2020. This is the author's version of the work. It is posted here by permission of the Authors for personal use, not for redistribution.

The final publication is available at IOS Press through http://dx.doi.org/10.3233/APC200077

Creative Commons License
Creative Commons Attribution-Noncommercial-No Derivative Works 4.0
Citation Information
Dario Dematties, George K. Thiruvathukal, Silvio Rizzi, Alejandro Wainselboim, et al.. "Towards High-End Scalability on Biologically-Inspired Computational Models" Parallel Computing: Technology Trends Vol. 36 (2020)
Available at: http://works.bepress.com/gkthiruvathukal/140/