标题：DDSN: Duplicate detection to reduce both storage and bandwidth consumption
作者：Zhang, Jiaran ;Yu, Xiaohui ;Liu, Yang ;Lin, Liwei
作者机构：[Zhang, Jiaran ;Yu, Xiaohui ;Liu, Yang ;Lin, Liwei ] School of Computer Science and Technology, Shandong University, Jinan, China;[Zhang, Jiaran ] Sof 更多
会议名称：2013 IEEE International Conference on Big Data, Big Data 2013
会议日期：6 October 2013 through 9 October 2013
来源：Proceedings - 2013 IEEE International Conference on Big Data, Big Data 2013
关键词：Duplicate Detection; Network File System
摘要：As highly centralized storage facilities are gaining popularity, duplicate detection becomes a critical problem. Traditional methods focus on reducing the storage space consumption; however, for network storage system with remote clients, the network overhead cannot be ignored, especially when the system is accessed over WAN. We propose a new duplicate detection method and implement a network file system prototype called DDSN based on this new method. It can reach the same performance in terms of storage space consumption as the state-of-the-art sliding blocking method. Meanwhile, our method overcomes its drawback that the whole file needs to be transmitted over the network, and therefore saves massive bandwidth for duplicate data. Experiments confirm the effectiveness of the proposed method. © 2013 IEEE.