基于MatLab的海量数据处理方法

Massive Data Process Methods Based on Matlab

MatLab是MathWorks公司推出的一种科学计算软件,在使用MatLab进行数据处理过程中,常常会用到循环语句逐点处理数据,当数据量巨大时,经常会出现耗时长久的情况。将循环运算转换为矩阵运算,充分利用MatLab高效率的矩阵运算特点,可以缩短数据处理时间。通过一个算例分析了创建大容量矩阵的方法:如扩大内存空间、采用短字节数值数据类型等。算例运行结果表明,通过分配更多的内存空间进行大容量矩阵运算,可以显著缩短数据处理的时间,从而实现空间换时间的目的。

Matlab is one kind of software for science computation manufactured by Mathworks Inc. Loop commands are usually used during data processing, and it often happens that processing duration is very long if the data amount is very large. Matrix computation is used to reduce data processing duration instead of loop computation in such case since matrix computation is very efficient in MatLab, but matrix computation needs more memory. Several ways to extend memory space are discussed via a computation example, and it verifies that less pro- cessing time is needed if more memory is allocated to data matrix in the data processing algorithm.