Design of quantum VQ iteration and quantum VQ encoding algorithm taking O（√N） steps for data compression

Vector quantization （VQ） is an important data compression method. The key of the encoding of VQ is to find the closest vector among N vectors for a feature vector. Many classical linear search algorithms take O（N） steps of distance computing between two vectors. The quantum VQ iteration and corresponding quantum VQ encoding algorithm that takes O（√N） steps are presented in this paper. The unitary operation of distance computing can be performed on a number of vectors simultaneously because the quantum state exists in a superposition of states. The quantum VQ iteration comprises three oracles, by contrast many quantum algorithms have only one oracle, such as Shor＇s factorization algorithm and Grover＇s algorithm. Entanglement state is generated and used, by contrast the state in Grover＇s algorithm is not an entanglement state. The quantum VQ iteration is a rotation over subspace, by contrast the Grover iteration is a rotation over global space. The quantum VQ iteration extends the Grover iteration to the more complex search that requires more oracles. The method of the quantum VQ iteration is universal.

The Compression Algorithm for the Data Acquisition System in HT-7 Tokamak

HT-7 superconducting tokamak in the Institute of Plasma Physics of the Chinese Academy of Sciences is an experimental device for fusion research in China. The main task of the data acquisition system of HT-7 is to acquire, store, analyze and index the data. The volume of the data is nearly up to hundreds of million bytes. Besides the hardware and software support, a great capacity of data storage, process and transfer is a more important problem. To deal with this problem, the key technology is data compression algorithm. In the paper, the data format in HT-7 is introduced first, then the data compression algorithm, LZO, being a kind of portable lossless data compression algorithm with ANSI C, is analyzed. This compression algorithm, which fits well with the data acquisition and distribution in the nuclear fusion experiment, offers a pretty fast compression and extremely fast decompression. At last the performance evaluation of LZO application in HT-7 is given.

A Ka-band Solid-state Transmitter Cloud Radar and Data Merging Algorithm for Its Measurements

This study concerns a Ka-band solid-state transmitter cloud radar, made in China, which can operate in three different work modes, with different pulse widths, and coherent and incoherent integration numbers, to meet the requirements for cloud remote sensing over the Tibetan Plateau. Specifically, the design of the three operational modes of the radar（i.e., boundary mode M1, cirrus mode M2, and precipitation mode M3） is introduced. Also, a cloud radar data merging algorithm for the three modes is proposed. Using one month＇s continuous measurements during summertime at Naqu on the Tibetan Plateau,we analyzed the consistency between the cloud radar measurements of the three modes. The number of occurrences of radar detections of hydrometeors and the percentage contributions of the different modes＇ data to the merged data were estimated.The performance of the merging algorithm was evaluated. The results indicated that the minimum detectable reflectivity for each mode was consistent with theoretical results. Merged data provided measurements with a minimum reflectivity of -35 dBZ at the height of 5 km, and obtained information above the height of 0.2 km. Measurements of radial velocity by the three operational modes agreed very well, and systematic errors in measurements of reflectivity were less than 2 dB. However,large discrepancies existed in the measurements of the linear depolarization ratio taken from the different operational modes.The percentage of radar detections of hydrometeors in mid- and high-level clouds increased by 60% through application of pulse compression techniques. In conclusion, the merged data are appropriate for cloud and precipitation studies over the Tibetan Plateau.

A Bit-level Text Compression Scheme Based on the ACW Algorithm

This paper presents a description and performance evaluation of a new bit-level, lossless, adaptive, and asymmetric data compression scheme that is based on the adaptive character wordlength （ACW（n）） algorithm. The proposed scheme enhances the compression ratio of the ACW（n） algorithm by dividing the binary sequence into a number of subsequences （s）, each of them satisfying the condition that the number of decimal values （d） of the n-bit length characters is equal to or less than 256. Therefore, the new scheme is referred to as ACW（n, s）, where n is the adaptive character wordlength and s is the number of subsequences. The new scheme was used to compress a number of text files from standard corpora. The obtained results demonstrate that the ACW（n, s） scheme achieves higher compression ratio than many widely used compression algorithms and it achieves a competitive performance compared to state-of-the-art compression tools.

A Survey and Tutorial of EEG-Based Brain Monitoring for Driver State Analysis

The driver’s cognitive and physiological states affect his/her ability to control the vehicle.Thus,these driver states are essential to the safety of automobiles.The design of advanced driver assistance systems(ADAS)or autonomous vehicles will depend on their ability to interact effectively with the driver.A deeper understanding of the driver state is,therefore,paramount.Electroencephalography(EEG)is proven to be one of the most effective methods for driver state monitoring and human error detection.This paper discusses EEG-based driver state detection systems and their corresponding analysis algorithms over the last three decades.First,the commonly used EEG system setup for driver state studies is introduced.Then,the EEG signal preprocessing,feature extraction,and classification algorithms for driver state detection are reviewed.Finally,EEG-based driver state monitoring research is reviewed in-depth,and its future development is discussed.It is concluded that the current EEGbased driver state monitoring algorithms are promising for safety applications.However,many improvements are still required in EEG artifact reduction,real-time processing,and between-subject classification accuracy.

A New Parallel-by-Cell Approach to Undistorted DataCompression Based on Cellular Automatonand Genetic Algorithm

In this paper, a new parallel-by-cell approach to the undistorteddata compression based on cellular automaton and genetic algorithm is presented.The local compression rules in a cellular automaton are obtained by using a geneticevolutionary algorithm. The correctness of the hyper-parallel compression, the timecomplexity, and the relevant symbolic dynamic behaviour are discussed. In comparison with other traditional sequential or small-scale parallel methods for undistorteddata compression, the proposed approach shows much higher real-time performance,better suitability and feasibility for the systolic hardware implementation.

A Survey of Bitmap Index Compression Algorithms for Big Data

With the growing popularity of Internet applications and the widespread use of mobile Internet, Internet traffic has maintained rapid growth over the past two decades. Internet Traffic Archival Systems（ITAS） for packets or flow records have become more and more widely used in network monitoring, network troubleshooting, and user behavior and experience analysis. Among the three key technologies in ITAS, we focus on bitmap index compression algorithm and give a detailed survey in this paper. The current state-of-the-art bitmap index encoding schemes include： BBC, WAH, PLWAH, EWAH, PWAH, CONCISE, COMPAX, VLC, DF-WAH, and VAL-WAH. Based on differences in segmentation, chunking, merge compress, and Near Identical（NI） features, we provide a thorough categorization of the state-of-the-art bitmap index compression algorithms. We also propose some new bitmap index encoding algorithms, such as SECOMPAX, ICX, MASC, and PLWAH＋, and present the state diagrams for their encoding algorithms. We then evaluate their CPU and GPU implementations with a real Internet trace from CAIDA. Finally, we summarize and discuss the future direction of bitmap index compression algorithms. Beyond the application in network security and network forensic, bitmap index compression with faster bitwise-logical operations and reduced search space is widely used in analysis in genome data, geographical information system, graph databases, image retrieval, Internet of things, etc. It is expected that bitmap index compression will thrive and be prosperous again in Big Data era since 1980s.

DNA Computing with Water Strider Based Vector Quantization for Data Storage Systems

The exponential growth of data necessitates an effective data storage scheme,which helps to effectively manage the large quantity of data.To accomplish this,Deoxyribonucleic Acid(DNA)digital data storage process can be employed,which encodes and decodes binary data to and from synthesized strands of DNA.Vector quantization(VQ)is a commonly employed scheme for image compression and the optimal codebook generation is an effective process to reach maximum compression efficiency.This article introduces a newDNAComputingwithWater StriderAlgorithm based Vector Quantization(DNAC-WSAVQ)technique for Data Storage Systems.The proposed DNAC-WSAVQ technique enables encoding data using DNA computing and then compresses it for effective data storage.Besides,the DNAC-WSAVQ model initially performsDNA encoding on the input images to generate a binary encoded form.In addition,aWater Strider algorithm with Linde-Buzo-Gray(WSA-LBG)model is applied for the compression process and thereby storage area can be considerably minimized.In order to generate optimal codebook for LBG,the WSA is applied to it.The performance validation of the DNAC-WSAVQ model is carried out and the results are inspected under several measures.The comparative study highlighted the improved outcomes of the DNAC-WSAVQ model over the existing methods.

Ensemble Deep Learning with Chimp Optimization Based Medical Data Classification

Eye state classification acts as a vital part of the biomedical sector,for instance,smart home device control,drowsy driving recognition,and so on.The modifications in the cognitive levels can be reflected via transforming the electro-encephalogram(EEG)signals.The deep learning(DL)models automated extract the features and often showcased improved outcomes over the conventional clas-sification model in the recognition processes.This paper presents an Ensemble Deep Learning with Chimp Optimization Algorithm for EEG Eye State Classifi-cation(EDLCOA-ESC).The proposed EDLCOA-ESC technique involves min-max normalization approach as a pre-processing step.Besides,wavelet packet decomposition(WPD)technique is employed for the extraction of useful features from the EEG signals.In addition,an ensemble of deep sparse autoencoder(DSAE)and kernel ridge regression(KRR)models are employed for EEG Eye State classification.Finally,hyperparameters tuning of the DSAE model takes place using COA and thereby boost the classification results to a maximum extent.An extensive range of simulation analysis on the benchmark dataset is car-ried out and the results reported the promising performance of the EDLCOA-ESC technique over the recent approaches with maximum accuracy of 98.50%.

基于Otsu的EEG通道选择情绪识别研究

脑电信号情绪识别是数据人机交互(HCI)技术的一种,实时情感识别对于模型性能要求较高,为实现以较低的运算成本获取较高的识别精度,采用时域滑动窗口的方法扩充样本量,基于Otsu算法筛选出含有最多情绪特征信息的通道,并利用快速傅里叶变换进行脑电信号频段提取,以功率谱密度作为特征,构建了基于支持向量机等分类模型,对高唤醒-低唤醒(HA-LA)和高效价-低效价(HV-LV)两种任务进行分类。实验表明,使用SVM分类器在HA-LA情绪识别任务中得到(82.2±0.4)%的识别准确率,在HV-LV情绪识别任务中得到(83.4±0.3)%的识别准确率。所提出的时域滑动窗口能有效提取含有情绪的脑电信号,在减少数据量的情况下仍获得了不错的情绪识别性能,为实时情感识别的脑机接口提供了一种高效的模型。

基于遗传优化聚类的GRU无损电力监测数据压缩

针对电力调度中心监测数据记录体量大、存储困难的问题,提出基于遗传优化K-means聚类的门控循环单元神经网络无损数据压缩方法。首先,搭建分布式集群,将多维原始电力数据聚类成相似性较高的数据块,并利用遗传算法对聚类进行寻优,提高数据聚类的效果;再通过门控循环单元神经网络训练数据编码的概率分布模型,结合算术编码对数据进行编码压缩;最后,以多个电力数据集为算例进行分析。经验证本文所提的压缩算法能实现数据的高比例压缩、优化集群性能。

基于SSA-BGOMP的滚动轴承振动信号压缩重构方法

针对广义正交匹配追踪算法(GOMP)在进行滚动轴承振动信号压缩感知重构的迭代过程中无法剔除错误原子,重构效果较差的问题,提出了基于麻雀搜索算法-回溯广义正交匹配追踪(SSA-BGOMP)的轴承振动信号压缩重构方法,在GOMP的基础上引入具有自适应特性的改进回溯机制,通过麻雀搜索算法自动设置阈值,对支撑集原子进行二次回溯筛选,从而降低错误原子选入支撑集的概率,提升算法的抗噪性和重构效果。仿真信号以及CWRU,XJTU-SY轴承故障数据集的试验结果表明:在DCT和K-SVD字典上,SSA-BGOMP比GOMP的相对误差分别降低2%~12%与3%~13%,有效改善了滚动轴承振动信号的压缩重构效果。

应用于脑机接口系统的动态稀疏矩阵压缩算法

在脑机接口系统中,高通道数神经信号采集是一个核心功能模块,能够为外部计算机设备采集大量人脑中的神经信息;在高通道数神经信号采集中,因其原始数据量巨大,直接传输和处理产生的原始数据会消耗极大的功耗并增加硬件设计上的难度;为解决这个问题,一个有效的方法是在数据传输和处理前依据原始神经信号数据的特点对其进行压缩;神经元动作电位信号具有不应期性即有效信号的时域宽度与信号重复周期之比很小;利用此特点,能够将多通道神经信号的数字标记输出在一定时间范围内定义为一个稀疏矩阵,并对此稀疏矩阵进行特征提取,根据其特征动态地采用优化算法进行数据压缩;所提出的算法在Xilinx平台使用FPGA进行设计与实现,并且将其作为中控硬件在32通道神经信号采集硬件系统上通过实时验证,实验证明提出的动态稀疏矩阵压缩算法可实现83.4%的数据压缩率。

基于时序数据压缩的大数据无损编码转换

针对当下时序数据压缩普遍存在压缩比小、压缩效率低的问题,进行基于时序数据压缩算法的海量大数据无损编码转换研究。该研究分为2部分,首先利用经验模态分解(EMD)算法对时序数据进行分解,分解为有效分量和噪声分量。其次,针对有效分量,利用Huffman算法进行压缩编码转换;针对噪声分量,利用LZ77算法进行压缩编码转换。实验结果表明:与3种传统压缩编码转换算法相比,该算法分别对Haptics和Phoneme数据集进行压缩的均方根失真度为3.854和3.624,压缩比为53.62%和47.85%,由此说明该算法更能够保证在不失真的前提下,以更快的速度完成数据压缩。

EEG无损压缩技术的研究

本文提出一种基于改进的DM编码和自适应二值算术编码的EEG无损压缩技术:首先对数据进行DM编码,然后再通过适当数据格式的转换以消除各个字节中的“冗余比特位”,同时整个数据基于比特位的熵值也因此降低;在此基础上,再对结果进行二值算术编码输出最终的压缩结果。经实践表明,该技术的压缩性能明显优越于常规的无损压缩算法,并且整个过程实现简单、实时性好。

基于大数据驱动技术的光通信信号判决方法

为识别光通信因自身限制和大气湍流影响产生的畸变信号,研究基于大数据驱动技术的光通信信号判决方法。大数据采集模块获取海量光通信信号数据,通过创建Kafka集群将存储的信号传输到核心服务层,采用压缩感知方法压缩接收到的信号,采用自排序熵获取信号特征,利用K均值算法实现光通信信号判决。实验结果表明:该方法所得压缩信号波形较为稀疏,且与原始波形变化趋势相同;不同光通信信号特征呈现的振动频谱差异较大,代表性优良;光通信信号判决性能理想,完全不受噪声数据的干扰。

基于耦合BAS-MLP的混凝土抗压强度预测

针对1 030组混凝土抗压强度试验数据,通过天牛须搜寻算法(BAS)来训练多层神经网络(MLP),并与混合复杂进化方法(SCE)-MLP、多元宇宙优化算法(MVO)-MLP这2种耦合模型算法进行对比分析,得到可用于预测混凝土抗压强度的算法模型.结果表明:BAS可以显著提高MLP的训练精度和预测精度,该算法比SCE-MLP、MVO-MLP耦合模型算法更快、更准确;与人工神经网络(ANN)和支持向量机(SVM)个体学习算法相比,元启发式算法在混凝土抗压强度预测方便表现出良好的优越性.同时讨论了BAS-MLP模型中与训练数据集数量和输入变量数量相关的因素,发现使用1 030组数据的80%即可获得良好的预测结果 .

基于改进旋转门算法的变电站数据压缩存储方法

为解决变电站数据采集与监视控制(supervisory control and data acquisition,SCADA)系统接入海量数据引发的数据存储问题,提出一种基于改进旋转门算法的变电站数据压缩存储方法。首先介绍了旋转门有损压缩算法,针对存储频率固定、门限值固定、忽视异常点等缺点,分别提出自适应变频数据存储策略、动态调整门限值策略和异常点记录策略以提高算法精度。其次,针对变电站SCADA系统遥信、遥控、遥调数据采用变位存储方法,遥测数据采用改进旋转门算法。最后,通过算例验证了所提方法的有效性。

一种基于SCD文件的合并单元高速数据压缩方法

在现代智能电网中,智能变电站安装了大量合并单元来同步发布电流互感器和电压互感器的暂态量,这些暂态数据有必要保存长达数年,从而覆盖设备生命周期,为设备状态维修、可靠性等研究提供原始信息支撑,但是如此长时与高频的海量数据给存储设备带来了巨大压力。文中首先将高频暂态数据分为固定不变的、状态变化的和周期变化的3种形式来进行预处理,将固定不变部分用SCD文件中的唯一标识代替,状态变化部分用事件记录文件代替,周期变化部分用SCD文件中双通道差量和周期差量来表示。然后使用16位哈夫曼完成最终压缩编码,并对比测试了各种预处理前后的压缩结果和不同编码的压缩结果。最终的测试结果表明该压缩方法比普通硬件压缩卡压缩比更大,压缩速率比普通压缩卡更快。

基于ARM的硬件压缩算法在Spark中的性能研究

鲲鹏920 CPU是2021年面世、全球第一款基于7纳米制造工艺的ARM 64位CPU,该CPU内置一个名为KAEzip的硬件加速引擎,其核心是一个硬件压缩算法,能通过硬件提升压缩与解压缩性能.相关研究表明,压缩算法的硬化与传统软件压缩算法相比具备明显性能优势.但大数据领域中的基础性系统软件都无法识别和使用这类算法.因此研究评估硬件压缩算法在大数据环境下的性能,发现揭示制约这类算法性能的关键因素以及可能存在的缺陷具有重要意义.为此,本文首先提出一种基于“生产-消费”模型的Spark任务性能模型,形式化地表示多维资源、压缩算法和Spark任务性能之间的内在关系,从理论上分析揭示出Spark下影响压缩算法性能的关键因素.然后提出一种三层架构支持Spark识别使用硬件压缩算法.这种分层架构为进一步调优硬件压缩算法在Spark中的性能提供了灵活性,也能复用到其他大数据系统软件.在此基础上本文以KAEzip为实验对象,使用经典Spark基准测试程序全面评估它在Spark中的性能,结合性能模型分析挖掘制约KAEzip性能的关键因素与根源.对KAEzip的测试表明:(1)硬件压缩算法可有效提升Spark性能。比如,KAEzip比snappy有最多13.8%的压缩性能优势、最多7%的解压优势和最多5.7%的实际应用场景下的性能优势;(2)磁盘的数据传输率与硬件压缩算法性能之间的不匹配是制约硬件压缩算法性能的重要因素;(3)压缩算法在Spark中的运行机制更易导致CPU的数据处理能力与硬件压缩算法性能不匹配,也制约着硬件压缩算法的性能.测试结果也表明KAEzip在压缩小数据时会导致数据膨胀问题.为此,本文扩展三层架构分析揭示出导致该问题的根源,并结合压缩算法在Spark中的运行机制提出一种优化方法.硬件压缩算法作为压缩算法领域的新研究方向,本文的研究工作不仅可广泛用于优化内置于CPU中的硬件压缩算法在Spark下的性能,也有助于持续演化完善KAEzip和鲲鹏920 CPU.