Skip to main content
Article
Hot-Starting Software Containers for STAR Aligner
GigaScience
  • Pai Zhang
  • Ling-Hong Hung
  • Wes Lloyd
  • Ka Yee Yeung, University of Washington Tacoma
Publication Date
8-1-2018
Document Type
Article
Abstract

Using software containers has become standard practice to reproducibly deploy and execute biomedical workflows on the cloud. However, some applications that contain time-consuming initialization steps will produce unnecessary costs for repeated executions. We demonstrate that hot-starting from containers that have been frozen after the application has already begun execution can speed up bioinformatics workflows by avoiding repetitive initialization steps. We use an open-source tool called Checkpoint and Restore in Userspace (CRIU) to save the state of the containers as a collection of checkpoint files on disk after it has read in the indices. The resulting checkpoint files are migrated to the host, and CRIU is used to regenerate the containers in that ready-to-run hot-start state. As a proof-of-concept example, we create a hot-start container for the spliced transcripts alignment to a reference (STAR) aligner and deploy this container to align RNA sequencing data. We compare the performance of the alignment step with and without checkpoints on cloud platforms using local and network disks. We demonstrate that hot-starting Docker containers from snapshots taken after repetitive initialization steps are completed significantly speeds up the execution of the STAR aligner on all experimental platforms, including Amazon Web Services, Microsoft Azure, and local virtual machines. Our method can be potentially employed in other bioinformatics applications in which a checkpoint can be inserted after a repetitive initialization phase.

DOI
10.1093/gigascience/giy092
Publisher Policy
open access
Citation Information
Pai Zhang, Ling-Hong Hung, Wes Lloyd and Ka Yee Yeung. "Hot-Starting Software Containers for STAR Aligner" GigaScience Vol. 7 Iss. 8 (2018)
Available at: http://works.bepress.com/ky-yeung/9/