Skip to main content
Dissertation
Flash-based Storage Management in Cloud Computing Datacenter Infrastructures
(2018)
  • Zhengyu Yang, Northeastern University
Abstract
A basic credendum of cloud computing can be summarized as: user devices are light terminals to assign jobs and gather results, while those heavy computations are conducted on remote distributed server clusters. This light-terminal-heavy-server structure makes high availability no longer an option, but a requirement in today’s datacenters. Furthermore, when bringing compute and storage capabilities into balance, we find that the biggest challenge here is closing the gap between compute and storage performance to shift storage’s curve back towards Moore’s law. In detail, the time consumed to wait for I/Os is the main cause of idling and wasting CPU resources, since a large number of popular cloud applications are I/O intensive, such as video streaming, file sync and backup, and data iteration for machine learning. Thus, storage I/O is the biggest bottleneck in large-scale datacenters. To address this bottleneck, Solid State Drives (SSDs) are widely being deployed as a per-virtual disk, second-level cache of Hard Disk Drives (HDDs) in SSD-HDD hybrid storage systems to improve I/O access performance, ascribing to SSD’s high I/O throughput, low I/O latency and low power consumption. Recently, the capital expenditure of flash-based SSDs keeps decreasing and the storage capacity of SSDs keeps increasing. Consequently, the “sales pitch” of traditional HDDs as backend storage – low cost and large capacity – is no longer unique, and eventually, these HDDs will be replaced by low-end SSDs. As a result, it is widely believed that all-flash multi-tier storage systems will be adopted in the enterprise datacenters running big data platforms in the near future. Therefore, in this dissertation, we focus on studying storage resource management for both SSD-HDD hybrid and all-flash storage systems, and investigating storage optimization for big data platforms. We aim to achieve high availability of storage capacity and accessibility by improving both performance and reliability in these storage systems.
Keywords
  • Flash
Disciplines
Publication Date
2018
Degree
Doctor of Philosophy
Field of study
Computer Engineering
Department
Electrical and Computer Engineering
Advisors
Ningfang Mi
Citation Information
Zhengyu Yang. "Flash-based Storage Management in Cloud Computing Datacenter Infrastructures" (2018)
Available at: http://works.bepress.com/zhengyuyang/34/