标题:A Self-Organizing Distributed Memory Cache for Data Sharing Applications in Cluster Environment
作者:Liu, Weifeng; Gong, Bin; Hu, Yi; Guo, Yiru
通讯作者:Liu, WF
作者机构:[Liu, Weifeng; Gong, Bin; Hu, Yi; Guo, Yiru] Shandong Univ, Sch Comp Sci & Technol, Jinan 250100, Shandong, Peoples R China.
会议名称:15th IEEE International Conference on High Performance Computing and Communications (HPCC) /11th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC)
会议日期:NOV 13-15, 2013
来源:2013 IEEE 15TH INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS & 2013 IEEE INTERNATIONAL CONFERENCE ON EMBEDDED AND UBIQUITOUS COMPUTING (HPCC_EUC)
出版年:2013
页码:158-164
DOI:10.1109/HPCC.and.EUC.2013.32
关键词:distributed memory cache; MPI; parallel I/O
摘要:Although capacities of persistent storage devices evolved rapidly in the last years, the bandwidth between memory and persistent storage devices is still the bottleneck. As loosely coupled data sharing applications running in cluster environment may need an enormous number of files, the access to these files might become the bottleneck. With the quick development of the server and high-speed network, there are many works done on distributed memory cache to minimize data requests to the centralized filesystem. These systems have the drawback that nodes are coupled together to form a distributed cache statically. It is a difficult administrative task for changing environments like clusters.; Current high performance computing resources, support batch job submissions using distributed resource management systems like TORQUE. How to use the resource management system to set up a self-organizing distributed memory cache on demand has rarely been studied. In this paper, we design a framework for dynamically setting up distributed memory cache for data sharing applications. Shared files are stored in the distributed memory cache, which can be accessed transparently and deliver data with high bandwidth. We describe the architecture of the framework, and evaluate its performance for a use case scenario.
收录类别:CPCI-S
资源类型:会议论文
TOP