基于改进SVD++算法和K-means++算法的小文件合并方案

A Small File Merging Scheme based on Improved SVD++Algorithm and Kmeans++Algorithm

文章提出了一种基于改进SVD++算法和K-means++算法的小文件合并方案。通过引入自适应学习率函数和基于并行分组的SVD++算法,优化了小文件的合并过程,以提高Hadoop存储小文件的效率。同时,利用K-means++算法对合并后的文件进行聚类,优化了数据存储方式,降低了存储空间的浪费。在Hadoop平台上进行的实验表明,该方案在保持数据处理准确性和稳定性的同时,显著提升了Hadoop存储与处理小文件的性能。

This paper proposes a small file merging scheme based on the improved SVD++algorithm and K-means++algorithm.By introducing an adaptive learning rate function and the parallel grouping based on the SVD++algorithm,the file merging process is optimized to enhance the efficiency of storing small files in Hadoop.Additionally,the K-means++algorithm is employed to cluster the merged files and optimize the data storage method to reduce storage space wastage.Experiments conducted on the Hadoop platform demonstrate that the proposed scheme significantly improves the performance of storing and processing small files while maintaining data processing accuracy and stability.