Resource Optimization for Speculative Execution in a MapReduce Cluster
Refereed conference paper presented and published in conference proceedings

香港中文大學研究人員

全文

引用次數

其它資訊
摘要The MapReduce paradigm is now the de facto standard for large-scale data analytics. In this paper we address the resource management issues in MapReduce Cluster. Speculative execution (task backup) plays an important role in resource management. We propose two different strategies and build two models to formulate the backup issue as an optimization problem when the cluster is lightly loaded. Moreover, we present an Enhanced Speculative Execution (ESE) algorithm when the cluster is heavily loaded and adopt the approximate analysis to get an optimal value for the parameter in the algorithm. The simulation results show that the algorithm can reduce the job completion time by 50% while consuming much less resource compared to the naive method without backup.
著者Xu HL, Lau WC
會議名稱21st IEEE International Conference on Network Protocols (ICNP)
會議開始日07.10.2013
會議完結日10.10.2013
會議地點Gottingen
會議國家/地區德國
出版年份2013
月份1
日期1
出版社IEEE
國際標準書號978-1-4799-1270-4
電子國際標準書號978-1-4799-1270-4
國際標準期刊號1092-1648
語言英式英語
關鍵詞job scheduling; MapReduce; speculative execution; theoretical analysis
Web of Science 學科類別Telecommunications

上次更新時間 2021-22-01 於 00:26