Attenuate Class Imbalance Problem for Pneumonia Diagnosis Using Ensemble Parallel Stacked Pre-Trained Models

Pneumonia is an acute lung infection that has caused many fatalitiesglobally. Radiologists often employ chest X-rays to identify pneumoniasince they are presently the most effective imaging method for this purpose.Computer-aided diagnosis of pneumonia using deep learning techniques iswidely used due to its effectiveness and performance. In the proposed method,the Synthetic Minority Oversampling Technique (SMOTE) approach is usedto eliminate the class imbalance in the X-ray dataset. To compensate forthe paucity of accessible data, pre-trained transfer learning is used, and anensemble Convolutional Neural Network (CNN) model is developed. Theensemble model consists of all possible combinations of the MobileNetv2,Visual Geometry Group (VGG16), and DenseNet169 models. MobileNetV2and DenseNet169 performed well in the Single classifier model, with anaccuracy of 94%, while the ensemble model (MobileNetV2+DenseNet169)achieved an accuracy of 96.9%. Using the data synchronous parallel modelin Distributed Tensorflow, the training process accelerated performance by98.6% and outperformed other conventional approaches.

Parallel Computing of a Variational Data Assimilation Model for GPS/MET Observation Using the Ray-Tracing Method

The Spectral Statistical Interpolation (SSI) analysis system of NCEP is used to assimilate meteorological data from the Global Positioning Satellite System (GPS/MET) refraction angles with the variational technique. Verified by radiosonde, including GPS/MET observations into the analysis makes an overall improvement to the analysis variables of temperature, winds, and water vapor. However, the variational model with the ray-tracing method is quite expensive for numerical weather prediction and climate research. For example, about 4 000 GPS/MET refraction angles need to be assimilated to produce an ideal global analysis. Just one iteration of minimization will take more than 24 hours CPU time on the NCEP's Cray C90 computer. Although efforts have been taken to reduce the computational cost, it is still prohibitive for operational data assimilation. In this paper, a parallel version of the three-dimensional variational data assimilation model of GPS/MET occultation measurement suitable for massive parallel processors architectures is developed. The divide-and-conquer strategy is used to achieve parallelism and is implemented by message passing. The authors present the principles for the code's design and examine the performance on the state-of-the-art parallel computers in China. The results show that this parallel model scales favorably as the number of processors is increased. With the Memory-IO technique implemented by the author, the wall clock time per iteration used for assimilating 1420 refraction angles is reduced from 45 s to 12 s using 1420 processors. This suggests that the new parallelized code has the potential to be useful in numerical weather prediction (NWP) and climate studies.

Pipeline synchronization in data parallel processing

Parallel processing has become an important way to further increase the computationpower of the computer system. It is obviously desirable that the parallel computers couldallow easy, efficient and flexible use of parallelism. Recently, technological factors areforcing a convergence towards parallel systems formed by a collection of essentially completecomputers connected by a communication network. This kind of network-based

A Granularity-Aware Parallel Aggregation Method for Data Streams

This paper focuses on the parallel aggregation processing of data streams based on the shared-nothing architecture. A novel granularity-aware parallel aggregating model is proposed. It employs parallel sampling and linear regression to describe the characteristics of the data quantity in the query window in order to determine the partition granularity of tuples, and utilizes equal depth histogram to implement partitio ning. This method can avoid data skew and reduce communi cation cost. The experiment results on both synthetic data and actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing.

Financial Data Modeling by Using Asynchronous Parallel Evolutionary Algorithms

In this paper, the high-level knowledge of financial data modeled by ordinary differential equations (ODEs) is discovered in dynamic data by using an asynchronous parallel evolutionary modeling algorithm (APHEMA). A numerical example of Nasdaq index analysis is used to demonstrate the potential of APHEMA. The results show that the dynamic models automatically discovered in dynamic data by computer can be used to predict the financial trends.

Regularized focusing inversion for large-scale gravity data based on GPU parallel computing

Processing large-scale 3-D gravity data is an important topic in geophysics field. Many existing inversion methods lack the competence of processing massive data and practical application capacity. This study proposes the application of GPU parallel processing technology to the focusing inversion method, aiming at improving the inversion accuracy while speeding up calculation and reducing the memory consumption, thus obtaining the fast and reliable inversion results for large complex model. In this paper, equivalent storage of geometric trellis is used to calculate the sensitivity matrix, and the inversion is based on GPU parallel computing technology. The parallel computing program that is optimized by reducing data transfer, access restrictions and instruction restrictions as well as latency hiding greatly reduces the memory usage, speeds up the calculation, and makes the fast inversion of large models possible. By comparing and analyzing the computing speed of traditional single thread CPU method and CUDA-based GPU parallel technology, the excellent acceleration performance of GPU parallel computing is verified, which provides ideas for practical application of some theoretical inversion methods restricted by computing speed and computer memory. The model test verifies that the focusing inversion method can overcome the problem of severe skin effect and ambiguity of geological body boundary. Moreover, the increase of the model cells and inversion data can more clearly depict the boundary position of the abnormal body and delineate its specific shape.

PORLES:A Parallel Object Relational Database System

We developed a parallel object relational DBMS named PORLES. It uses BSP model as its parallel computing model, and monoid calculus as its basis of data model. In this paper, we introduce its data model, parallel query optimization, transaction processing system and parallel access method in detail.

基于相对熵和余弦相似度的并行SVM算法

针对大数据环境下并行支持向量机(SVM)算法存在子集分布偏差大,并行效率低以及过滤非支持向量不准确等问题,提出了基于相对熵和余弦相似度的并行SVM算法(RC-PSVM)。该算法首先提出基于相对熵的数据划分策略(DPRE),平衡当前子集和原始数据集的相对熵,划分样本到适合的子集,降低子集分布偏差;然后提出基于余弦相似度的冗余层级检测策略(CS-RLDS),计算相邻层局部SVM之间法向量的余弦相似度,比较设定的阈值与相似度,识别并停止冗余层级,提高了并行效率;最后提出非支持向量过滤策略(NSVF),结合样本到多个局部支持向量模型决策边界的距离,计算支持向量相似度来识别非支持向量,解决了过滤非支持向量不准确的问题。实验表明,RC-PSVM算法的分类效果更佳,且在大数据下的运行效率更高。

面向数据湖存取性能优化的数据并行处理技术研究

本文围绕数据湖应用背景下海量数据的高性能存取需求,以新型数据存储模型和分布式存储及缓存机制为目标,通过对数据湖存储结构、数据访问模式和数据处理方法进行分析,开展数据湖存取性能优化问题研究。首先,结合数据湖系统中的文件系统存储方式,设计了一种基于列式存储的数据存储结构,并通过索引优化技术提高数据访问速度。其次,针对数据湖中常用的批处理和流处理这两种访问模式,提出了一种基于数据分区和缓存机制的访问优化方案,以提高数据访问的效率和稳定性。最后,针对数据湖在大规模数据情况下更新和增量计算出现的计算时间长问题,提出了一种基于Spark并行计算和分布式文件系统(Hadoop Distributed File System,HDFS)的数据处理方案,以提高数据处理的速度和可靠性。实验结果表明,本文提出的数据湖存储性能优化技术相对于现有方法能够有效地提高数据的存储、访问和处理效率。

MapReduce模型在大规模数据并行挖掘中的应用

MapReduce并行编程模型通过定义良好的接口和运行支持库,能够自动并行执行大规模计算任务,隐藏底层实现细节,降低并行编程的难度。系统阐述MapReduce的基本工作原理及其工作流程,以TeraSort算法为例,针对其存在的问题,提出动态数据分区和数据压缩等优化建议。研究成果表明,优化后的TeraSort算法能够显著缩短数据处理时间,优化系统的吞吐量,并改善资源分配的均衡性。

One-End Data Method for Fault Position Estimate of Two-Parallel Transmission Lines

An accurate numerical algorithm for three-line fault involving different phases from each of two-parallel lines is presented. It is based on one-terminal voltage and current data. The loop and nodel equations comparing faulted phase to non-faulted phase of two-parallel lines are introduced in the fault location estimation modal, in which the faulted impedance of remote end is not involved. The effect of load flow and fault resistance on the accuracy of fault location are effectively eliminated, therefore an accurate algorithm of locating fault is derived. The algorithm is demonstrated by digital computer simulations and the results show that errors in locating fault are less than 1%.

数据驱动模式下基于非平行超平面SVM算法的贸易经济预测

数据驱动的多元化发展导致数据异构性增强、维度提升和特征量规模扩大,给贸易经济分析带来更大挑战。为了提高贸易经济分析的科学性,采用非平行超平面支持向量机算法(support vector machine,SVM)对贸易经济进行预测分析。首先,根据贸易经济影响因素进行主成分分析,获取影响贸易经济的关键特征,并对特征进行量化和去噪处理。然后,采用广义特征值最接近支持向量机(proximal support vector machine via generalized eigenvalues,GEPSVM)进行贸易经济预测分类。根据预测指标要求,选择核函数GEPSVM算法(KGEPSVM算法)对分类的非平行超平面求解,通过类别划分函数获得经济预测结果。实证分析表明,对比常用的非平行超平面支持向量机算法,所提算法的贸易经济预测性能更优,而且在常用贸易经济指标的预测中,表现出较高预测精度和稳定性。

大数据场景下用户评论聚类文本挖掘算法

因传统文本数据挖掘算法在大数据场景下的文本聚类挖掘效果较差,提出一种大数据场景下基于文本数据挖掘的用户评论聚类算法。首先,通过设计改进的信息增益算法提取用户评论数据特征,根据信息熵提取文本关键字和不平衡数据项形成特征数据。之后,使用改进的聚类数据挖掘算法对特征数据进行聚类挖掘。最后,基于Spark框架将改进的聚类数据挖掘算法进行并行化改造。设计实验验证分析所提特征提取算法与聚类挖掘算法的性能,结果表明在大数据场景下所提算法的运行时间、准确率和加速比方面优于传统算法。

面向大数据的可扩展正则采样并行排序算法

排序算法是计算机科学领域的一个基础算法,是大量应用的算法核心。在大数据时代,随着数据量的极速增长,并行排序算法受到广泛关注。现有的并行排序算法普遍存在通信开销过大、负载不均衡等问题,导致算法难以大规模扩展。针对以上问题,提出一种大规模可扩展的正则采样并行排序(scalable parallel sorting by regular sampling,ScaPSRS)算法,摒弃传统正则采样并行排序(parallel sorting by regular sampling,PSRS)算法中由一个进程负责采样的做法,转而让所有进程参与正则采样,选出p-1个分隔元素,将整个数据集划分成p个不相交的子集,然后实施并行排序,避免了单一进程的采样瓶颈。此外,ScaPSRS采用一种新的迭代更新策略选择p-1个分隔元素,保证划分的p个子集尽可能大小相同,从而确保p个进程对各自的子集进行本地排序时的负载均衡。在天河二号超级计算机上进行的大量实验表明,ScaPSRS算法能够成功地扩展到32000个内核,性能比PSRS算法和Hofmann等人提出的分区算法分别提升了3.7倍和11.7倍。

基于流计算和大数据平台的实时交通流预测

目前交通流预测实时性差,很难满足在线分析和预测任务的需求,基于此提出一种Flink流计算框架和大数据平台结合的实时交通流预测方法。基于流计算框架实时捕捉和预处理数据,包括采用Flink的transform算子对数据进行校验和处理,将处理后的数据sink到大数据的HDFS文件系统,交由下一步的大数据并行框架进行分析建模与训练,实现基于流计算和大数据平台的实时交通流预测。实验结果表明,Flink能够实时捕捉和预处理交通流数据,把数据准时无误送入分布式文件系统中,在此基础上借助大数据框架下的并行分析和建模优势,在实时性数据分析与预测方面取得了较好的效果。

Hadoop平台下大学生学习预警模型研究

将大数据技术应用到教育领域,利用高校长期积累的海量数据,预警大学生可能存在的学习问题。首先给出数据特征提取方法,提取数据特征,得出特征集合;其次提出特征选择方法,根据特征集合选择适合大学生学习预警的特征;然后根据教育大数据的特点和学习预警的需求,选用加权投票组合预警方法组合分类回归树、逻辑回归和贝叶斯等单一模型;最后为满足大数据处理的要求,采用Hadoop平台并行化处理组合预警模型,构建大学生学习预警模型。通过模型评估,证明该模型的准确率、灵敏度和特异性能满足学习预警要求。

用于VSLAM系统的CNN在FPGA平台上的加速

为实现视觉同步定位与建图系统中卷积神经网络在FPGA上的加速,基于SuperPoint模型设计一种低功耗高效CNN加速器及相应的SoC系统。采用循环分块、数据复用、计算单元展开和双缓冲策略充分利用加速器的片上资源;为提高突发传输效率,预先对权重参数重排;提出Pack模块和Unpack模块,设计多通道数据传输,用于提高传输带宽。在Ultra96-V2 FPGA平台上部署整个SoC系统,在仅3 W左右的功耗下实现25.63 GOPS的吞吐量,其BRAM效率、DSP效率、性能密度和功耗效率相比之前的文献有明显优势。

面向分布式数据库的算子并行优化策略

随着网络技术的不断发展,数据规模呈现爆发式增长,使得传统的单机数据库逐步被分布式数据库所取代。分布式数据库采用节点协同工作方式解决了大规模数据存储问题,但由于增加了节点间通信开销,查询效率却不如单机数据库。分布式架构下,存储节点的数据仅用作多备份的冗余,为系统故障时提供数据恢复,并未被利用起来改善查询效率。针对上述问题,提出了一种面向分布式数据库的算子并行优化策略,通过对关键物理算子进行拆分,将拆分后的子请求均匀分配到存储层多个节点,由多个节点并行处理,从而减少查询响应时间。上述策略已经在分布式数据库CBase上进行了应用,实验表明,提出的并行优化策略可显著缩短SQL请求查询时间,并提高系统资源利用率。