Parallel Computing of a Variational Data Assimilation Model for GPS/MET Observation Using the Ray-Tracing Method

The Spectral Statistical Interpolation (SSI) analysis system of NCEP is used to assimilate meteorological data from the Global Positioning Satellite System (GPS/MET) refraction angles with the variational technique. Verified by radiosonde, including GPS/MET observations into the analysis makes an overall improvement to the analysis variables of temperature, winds, and water vapor. However, the variational model with the ray-tracing method is quite expensive for numerical weather prediction and climate research. For example, about 4 000 GPS/MET refraction angles need to be assimilated to produce an ideal global analysis. Just one iteration of minimization will take more than 24 hours CPU time on the NCEP's Cray C90 computer. Although efforts have been taken to reduce the computational cost, it is still prohibitive for operational data assimilation. In this paper, a parallel version of the three-dimensional variational data assimilation model of GPS/MET occultation measurement suitable for massive parallel processors architectures is developed. The divide-and-conquer strategy is used to achieve parallelism and is implemented by message passing. The authors present the principles for the code's design and examine the performance on the state-of-the-art parallel computers in China. The results show that this parallel model scales favorably as the number of processors is increased. With the Memory-IO technique implemented by the author, the wall clock time per iteration used for assimilating 1420 refraction angles is reduced from 45 s to 12 s using 1420 processors. This suggests that the new parallelized code has the potential to be useful in numerical weather prediction (NWP) and climate studies.

POTENTIAL: A Highly Adaptive Core of Parallel Database System

POTENTIAL is a virtual database machine based on general computing platforms, especially parallel computing platforms. It provides a complete solution to high-performance database systems by a 'virtual processor + virtual data bus + virtual memory' architecture. Virtual processors manage all CPU resources in the system, on which various operations are running. Virtual data bus is responsible for the management of data transmission between associated operations, which forms the hinges of the entire system. Virtual memory provides efficient data storage and buffering mechanisms that conform to data reference behaviors in database systems. The architecture of POTENTIAL is very clear and has many good features, including high efficiency, high scalability, high extensibility, high portability, etc.

An Improved Hilbert Curve for Parallel Spatial Data Partitioning

一条新奇 Hilbert 曲线为划分的平行空间数据被介绍，与空间信息和向量数据项的可变长度的特征的巨大数量的性质的考虑。基于改进 Hilbert 弯曲，算法能被设计空间数据在平行空间数据库在多重磁盘之中划分完成几乎制服。因此，数据不平衡的现象能显著地被避免，搜索和询问效率能被提高。

A Granularity-Aware Parallel Aggregation Method for Data Streams

This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and linear regression to describe the characteristics of the data quantity in the query window in order to determine the partition granularity of tuples, and utilizes equal depth histogram to implement partitio ning. This method can avoid data skew and reduce communi cation cost. The experiment results on both synthetic data and actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing.

Parallel Data Cube Storage Structure for Range Sum Queries and Dynamic Updates

I/O parallelism is considered to be a promising approach to achieving highperformance in parallel data warehousing systems where huge amounts of data and complex analyticalqueries have to be processed. This paper proposes a parallel secondary data cube storage structure(PHC for short) to efficiently support the processing of range sum queries and dynamic updates ondata cube using parallel computing systems. Based on PHC, two parallel algorithms for processingrange sum queries and updates are proposed also. Both the algorithms have the same time complexity,O(log^d n/P). The analytical and experimental results show that PHC and the parallel algorithms havehigh performance and achieve optimum speedup.

Attenuate Class Imbalance Problem for Pneumonia Diagnosis Using Ensemble Parallel Stacked Pre-Trained Models

Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Computer-aided diagnosis of pneumonia using deep learning techniques iswidely used due to its effectiveness and performance. In the proposed method,the Synthetic Minority Oversampling Technique (SMOTE) approach is usedto eliminate the class imbalance in the X-ray dataset. To compensate forthe paucity of accessible data, pre-trained transfer learning is used, and anensemble Convolutional Neural Network (CNN) model is developed. Theensemble model consists of all possible combinations of the MobileNetv2,Visual Geometry Group (VGG16), and DenseNet169 models. MobileNetV2and DenseNet169 performed well in the Single classifier model, with anaccuracy of 94%, while the ensemble model (MobileNetV2+DenseNet169)achieved an accuracy of 96.9%. Using the data synchronous parallel modelin Distributed Tensorflow, the training process accelerated performance by98.6% and outperformed other conventional approaches.

Financial Data Modeling by Using Asynchronous Parallel Evolutionary Algorithms

In this paper, the high-level knowledge of financial data modeled by ordinary differential equations (ODEs) is discovered in dynamic data by using an asynchronous parallel evolutionary modeling algorithm (APHEMA). A numerical example of Nasdaq index analysis is used to demonstrate the potential of APHEMA. The results show that the dynamic models automatically discovered in dynamic data by computer can be used to predict the financial trends.

Regularized focusing inversion for large-scale gravity data based on GPU parallel computing

Processing large-scale 3-D gravity data is an important topic in geophysics field. Many existing inversion methods lack the competence of processing massive data and practical application capacity. This study proposes the application of GPU parallel processing technology to the focusing inversion method, aiming at improving the inversion accuracy while speeding up calculation and reducing the memory consumption, thus obtaining the fast and reliable inversion results for large complex model. In this paper, equivalent storage of geometric trellis is used to calculate the sensitivity matrix, and the inversion is based on GPU parallel computing technology. The parallel computing program that is optimized by reducing data transfer, access restrictions and instruction restrictions as well as latency hiding greatly reduces the memory usage, speeds up the calculation, and makes the fast inversion of large models possible. By comparing and analyzing the computing speed of traditional single thread CPU method and CUDA-based GPU parallel technology, the excellent acceleration performance of GPU parallel computing is verified, which provides ideas for practical application of some theoretical inversion methods restricted by computing speed and computer memory. The model test verifies that the focusing inversion method can overcome the problem of severe skin effect and ambiguity of geological body boundary. Moreover, the increase of the model cells and inversion data can more clearly depict the boundary position of the abnormal body and delineate its specific shape.

PORLES:A Parallel Object Relational Database System

We developed a parallel object relational DBMS named PORLES. It uses BSP model as its parallel computing model, and monoid calculus as its basis of data model. In this paper, we introduce its data model, parallel query optimization, transaction processing system and parallel access method in detail.

One-End Data Method for Fault Position Estimate of Two-Parallel Transmission Lines

An accurate numerical algorithm for three-line fault involving different phases from each of two-parallel lines is presented. It is based on one-terminal voltage and current data. The loop and nodel equations comparing faulted phase to non-faulted phase of two-parallel lines are introduced in the fault location estimation modal, in which the faulted impedance of remote end is not involved. The effect of load flow and fault resistance on the accuracy of fault location are effectively eliminated, therefore an accurate algorithm of locating fault is derived. The algorithm is demonstrated by digital computer simulations and the results show that errors in locating fault are less than 1%.

MapReduce模型在大规模数据并行挖掘中的应用

MapReduce并行编程模型通过定义良好的接口和运行支持库,能够自动并行执行大规模计算任务,隐藏底层实现细节,降低并行编程的难度。系统阐述MapReduce的基本工作原理及其工作流程,以TeraSort算法为例,针对其存在的问题,提出动态数据分区和数据压缩等优化建议。研究成果表明,优化后的TeraSort算法能够显著缩短数据处理时间,优化系统的吞吐量,并改善资源分配的均衡性。

面向大数据的可扩展正则采样并行排序算法

排序算法是计算机科学领域的一个基础算法,是大量应用的算法核心。在大数据时代,随着数据量的极速增长,并行排序算法受到广泛关注。现有的并行排序算法普遍存在通信开销过大、负载不均衡等问题,导致算法难以大规模扩展。针对以上问题,提出一种大规模可扩展的正则采样并行排序(scalable parallel sorting by regular sampling,ScaPSRS)算法,摒弃传统正则采样并行排序(parallel sorting by regular sampling,PSRS)算法中由一个进程负责采样的做法,转而让所有进程参与正则采样,选出p-1个分隔元素,将整个数据集划分成p个不相交的子集,然后实施并行排序,避免了单一进程的采样瓶颈。此外,ScaPSRS采用一种新的迭代更新策略选择p-1个分隔元素,保证划分的p个子集尽可能大小相同,从而确保p个进程对各自的子集进行本地排序时的负载均衡。在天河二号超级计算机上进行的大量实验表明,ScaPSRS算法能够成功地扩展到32000个内核,性能比PSRS算法和Hofmann等人提出的分区算法分别提升了3.7倍和11.7倍。

面向分布式数据库的算子并行优化策略

随着网络技术的不断发展,数据规模呈现爆发式增长,使得传统的单机数据库逐步被分布式数据库所取代。分布式数据库采用节点协同工作方式解决了大规模数据存储问题,但由于增加了节点间通信开销,查询效率却不如单机数据库。分布式架构下,存储节点的数据仅用作多备份的冗余,为系统故障时提供数据恢复,并未被利用起来改善查询效率。针对上述问题,提出了一种面向分布式数据库的算子并行优化策略,通过对关键物理算子进行拆分,将拆分后的子请求均匀分配到存储层多个节点,由多个节点并行处理,从而减少查询响应时间。上述策略已经在分布式数据库CBase上进行了应用,实验表明,提出的并行优化策略可显著缩短SQL请求查询时间,并提高系统资源利用率。

大数据场景下用户评论聚类文本挖掘算法

因传统文本数据挖掘算法在大数据场景下的文本聚类挖掘效果较差,提出一种大数据场景下基于文本数据挖掘的用户评论聚类算法。首先,通过设计改进的信息增益算法提取用户评论数据特征,根据信息熵提取文本关键字和不平衡数据项形成特征数据。之后,使用改进的聚类数据挖掘算法对特征数据进行聚类挖掘。最后,基于Spark框架将改进的聚类数据挖掘算法进行并行化改造。设计实验验证分析所提特征提取算法与聚类挖掘算法的性能,结果表明在大数据场景下所提算法的运行时间、准确率和加速比方面优于传统算法。

基于并行双通道时空网络的流量数据修复技术

流量数据丢失是网络系统中常见的问题,通常由传感器故障、传输错误和存储丢失引起.现有的数据修复方法无法学习流量数据的多维特征,因此本文提出了一种结合双向长短期记忆网络与多尺度卷积网络的双通道并行架构(ST-MFCN)用于填补流量数据的缺失值,同时设计了一种新的对抗性损失函数进一步提高预测精度,该模型有效地学习流量数据的时间特征和动态空间特征.本文在Web traffic time series数据集上对模型进行测试,并与现有的修复方法进行对比,实验结果表明,ST-MFCN能够减少数据恢复的误差,提升了数据修复的精确度,为网络系统中的流量数据修复提供了一种稳健高效的解决方案.

超大规模数据处理中并行计算技术的应用研究

随着人工智能和大数据时代的到来,超大规模数据处理成了一个重要的研究领域。该文主要探讨并行计算技术在超大规模数据处理中的应用,首先详细阐述并行计算和超大规模数据处理的基本理论与概念,特别是并行计算的编程模型与工具,最后通过分析并行计算在搜索引擎、气象预报和金融分析等中的实际案例,阐述并行计算技术在超大规模数据处理中的实际应用。

基于大数据挖掘的城市规划异构数据调度平台

为了提高城市规划异构数据资源的调度效果,设计基于大数据挖掘的城市规划异构数据调度平台。通过配置FPGA采集异构大数据;采用大数据挖掘技术中的二进制分裂聚类算法分类存储数据;基于粒子群聚类算法分发匹配调度任务;基于大数据处理技术优化数据调度效率,完成城市规划异构数据资源调度。测试结果显示:该平台的数据采集速度均在6800字节/s以上;弗里德曼检验和调整随机误差结果最高值分别为0.965和0.02,数据分类效果良好;数据调度匹配率均在0.94以上,提高了城市规划异构数据资源调度效果。