Surface modeling with very large data sets is challenging. An efficient method for modeling massive data sets using the high accuracy surface modeling method(HASM) is proposed, and HASM_Big is developed to handle very...Surface modeling with very large data sets is challenging. An efficient method for modeling massive data sets using the high accuracy surface modeling method(HASM) is proposed, and HASM_Big is developed to handle very large data sets. A large data set is defined here as a large spatial domain with high resolution leading to a linear equation with matrix dimensions of hundreds of thousands. An augmented system approach is employed to solve the equality-constrained least squares problem(LSE) produced in HASM_Big, and a block row action method is applied to solve the corresponding very large matrix equations.A matrix partitioning method is used to avoid information redundancy among each block and thereby accelerate the model.Experiments including numerical tests and real-world applications are used to compare the performances of HASM_Big with its previous version, HASM. Results show that the memory storage and computing speed of HASM_Big are better than those of HASM. It is found that the computational cost of HASM_Big is linearly scalable, even with massive data sets. In conclusion,HASM_Big provides a powerful tool for surface modeling, especially when there are millions or more computing grid cells.展开更多
基金supported by the National Natural Science Foundation of China (Grant Nos. 41541010, 41701456, 41421001, 41590840 & 91425304)the Key Programs of the Chinese Academy of Sciences (Grant No. QYZDY-SSW-DQC007)the Cultivate Project of Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences (Grant No. TSYJS03)
文摘Surface modeling with very large data sets is challenging. An efficient method for modeling massive data sets using the high accuracy surface modeling method(HASM) is proposed, and HASM_Big is developed to handle very large data sets. A large data set is defined here as a large spatial domain with high resolution leading to a linear equation with matrix dimensions of hundreds of thousands. An augmented system approach is employed to solve the equality-constrained least squares problem(LSE) produced in HASM_Big, and a block row action method is applied to solve the corresponding very large matrix equations.A matrix partitioning method is used to avoid information redundancy among each block and thereby accelerate the model.Experiments including numerical tests and real-world applications are used to compare the performances of HASM_Big with its previous version, HASM. Results show that the memory storage and computing speed of HASM_Big are better than those of HASM. It is found that the computational cost of HASM_Big is linearly scalable, even with massive data sets. In conclusion,HASM_Big provides a powerful tool for surface modeling, especially when there are millions or more computing grid cells.
