一种支持数据去冗和扩容的多媒体文件云存储系统实现

A Multimedia File Cloud Storage System to Support Data Deduplication and Logical Expansion

随着互联网的发展,存储多媒体文件的场景日益增多,云存储系统成为了业界焦点,很多云存储系统为应用程序提供数据存储、查询和计算服务.许多应用程序拥有大量重复的体积较小的多媒体文件,传统的分布式文件系统已不能满足多媒体文件对存储性能的需求,它们通常会把多媒体文件分块存储在不同的存储服务器上,每次获取文件内容都需要从多个块结点把数据获取到,然后重构文件内容给应用程序,导致访问这类体积较小的多媒体文件会消耗更多的资源.为了解决数据冗余度高的应用程序在低存储成本条件下对多媒体文件进行分布式存储的问题,同时为了提高这类应用程序的运行效率,首先提出一种分布式存储目录建模方案,可以用来描述数据中心的存储目录逻辑结构;然后实现了一个多媒体文件云存储系统MFCSS,支持数据去冗和存储目录逻辑扩容.分析和实验结果表明:MFCSS系统在保存数据冗余度较高的多媒体文件时具有良好的性能,可以有效提高磁盘的存储效率,同时具备良好的扩展性,可以简化应用程序管理分布式存储环境中多媒体文件的过程.

With the development of the Internet,the scene of storing multimedia files is increasing and cloud storage system has become the focus of the Internet field.Many cloud storage systems provide applications with data storage,query and computing service.A lot of applications have a large number of small-sized and duplicate multimedia files.Traditional distributed file systems are not suitable for storing and accessing multimedia files.Because in these systems,multimedia files are usually divided into several blocks which are stored on many data servers.Every time when applications need to get the file content,these file systems need reconstructing.This strategy can cause the problem of consuming more resources when users access the multimedia files.In order to let these applications with high data redundancy store multimedia files efficiently at a low cost,we propose a model of distributed directory tree to describe the logic structure of directories in the data center.Afterward,we design and implement a distributed multimedia file system named MFCSS which can support data deduplication and the logical expansion for directories.The experiment results show that the system not only has good performance when it saves multimedia files with higher redundancy but also can effectively improve the efficiency of disk storage.Moreover,the MFCSS system has good scalability and can simplify the process of managing multimedia files stored in distributed storage environment for applications.