Enabling Concurrent Failure Recovery for Regenerating-Coding-Based Storage Systems: From Theory to Practice
Publication in refereed journal

Times Cited
Web of Science17WOS source URL (as at 11/08/2020) Click here for the latest count
Altmetrics Information

Other information
AbstractData availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data of failed nodes. This paper explores recovery solutions based on regenerating codes, which have been designed to provide fault-tolerant storage and minimum bandwidth. Existing optimal regenerating codes are designed for single node failures. We build a system called CORE, which augments existing optimal regenerating codes for the recovery of a general number of failures including single and concurrent failures. We show theoretically that CORE achieves the minimum possible bandwidth for most cases. We implement a CORE prototype and evaluate it atop an HDFS cluster testbed with up to 20 storage nodes. We demonstrate that our CORE prototype conforms to our theoretical findings and achieves bandwidth savings when compared to the conventional recovery approach based on erasure codes.
All Author(s) ListLi RH, Lin J, Lee PPC
Journal nameIEEE Transactions on Computers
Volume Number64
Issue Number7
Pages1898 - 1911
LanguagesEnglish-United Kingdom
Keywordscoding theory; distributed storage systems; experiments and implementation; failure recovery; Regenerating codes
Web of Science Subject CategoriesComputer Science; Computer Science, Hardware & Architecture; Engineering; Engineering, Electrical & Electronic

Last updated on 2020-12-08 at 00:19