Design of quantum VQ iteration and quantum VQ encoding algorithm taking O（√N） steps for data compression

Vector quantization （VQ） is an important data compression method. The key of the encoding of VQ is to find the closest vector among N vectors for a feature vector. Many classical linear search algorithms take O（N） steps of distance computing between two vectors. The quantum VQ iteration and corresponding quantum VQ encoding algorithm that takes O（√N） steps are presented in this paper. The unitary operation of distance computing can be performed on a number of vectors simultaneously because the quantum state exists in a superposition of states. The quantum VQ iteration comprises three oracles, by contrast many quantum algorithms have only one oracle, such as Shor＇s factorization algorithm and Grover＇s algorithm. Entanglement state is generated and used, by contrast the state in Grover＇s algorithm is not an entanglement state. The quantum VQ iteration is a rotation over subspace, by contrast the Grover iteration is a rotation over global space. The quantum VQ iteration extends the Grover iteration to the more complex search that requires more oracles. The method of the quantum VQ iteration is universal.

RESEARCH ON ADAPTIVE DATA COMPRESSION METHOD FOR TRIANGULATED SURFACES

NC code or STL file can be generated directly from measuring data in a fastreverse-engineering mode. Compressing the massive data from laser scanner is the key of the newmode. An adaptive compression method based on triangulated-surfaces model is put forward.Normal-vector angles between triangles are computed to find prime vertices for removal. Ring datastructure is adopted to save massive data effectively. It allows the efficient retrieval of allneighboring vertices and triangles of a given vertices. To avoid long and thin triangles, a newre-triangulation approach based on normalized minimum-vertex-distance is proposed, in which thevertex distance and interior angle of triangle are considered. Results indicate that the compressionmethod has high efficiency and can get reliable precision. The method can be applied in fastreverse engineering to acquire an optimal subset of the original massive data.

DPCM-based vibration sensor data compression and its effect on structural system identification

Due to the large scale and complexity of civil infrastructures, structural health monitoring typically requires a substantial number of sensors, which consequently generate huge volumes of sensor data. Innovative sensor data compression techniques are highly desired to facilitate efficient data storage and remote retrieval of sensor data. This paper presents a vibration sensor data compression algorithm based on the Differential Pulse Code Modulation (DPCM) method and the consideration of effects of signal distortion due to lossy data compression on structural system identification. The DPCM system concerned consists of two primary components: linear predictor and quantizer. For the DPCM system considered in this study, the Least Square method is used to derive the linear predictor coefficients and Jayant quantizer is used for scalar quantization. A 5-DOF model structure is used as the prototype structure in numerical study. Numerical simulation was carried out to study the performance of the proposed DPCM-based data compression algorithm as well as its effect on the accuracy of structural identification including modal parameters and second order structural parameters such as stiffness and damping coefficients. It is found that the DPCM-based sensor data compression method is capable of reducing the raw sensor data size to a significant extent while having a minor effect on the modal parameters as well as second order structural parameters identified from reconstructed sensor data.

A Distributed Covert Channel of the Packet Ordering Enhancement Model Based on Data Compression

Covert channel of the packet ordering is a hot research topic.Encryption technology is not enough to protect the security of both sides of communication.Covert channel needs to hide the transmission data and protect content of communication.The traditional methods are usually to use proxy technology such as tor anonymous tracking technology to achieve hiding from the communicator.However,because the establishment of proxy communication needs to consume traffic,the communication capacity will be reduced,and in recent years,the tor technology often has vulnerabilities that led to the leakage of secret information.In this paper,the covert channel model of the packet ordering is applied into the distributed system,and a distributed covert channel of the packet ordering enhancement model based on data compression(DCCPOEDC)is proposed.The data compression algorithms are used to reduce the amount of data and transmission time.The distributed system and data compression algorithms can weaken the hidden statistical probability of information.Furthermore,they can enhance the unknowability of the data and weaken the time distribution characteristics of the data packets.This paper selected a compression algorithm suitable for DCCPOEDC and analyzed DCCPOEDC from anonymity,transmission efficiency,and transmission performance.According to the analysis results,it can be seen that DCCPOEDC optimizes the covert channel of the packet ordering,which saves the transmission time and improves the concealment compared with the original covert channel.

An Information Theory Approach to the Data Compression and Imaging System for Synthetic Aperture Radar （SAR）

Synthetic aperture radar (SAR) is portrayed as a multiple access channel. An information theory approach is applied to the SAR imaging system, and the information content about a target that can be extracted from its radar image is evaluated by the average mutual information measure. A conditional (transition) probability density function (PDF) of the SAR imaging system is derived by analyzing the system and a closed form of the information content is found. It is shown that the information content obtained by the SAR imaging system from an independent sample of echoes will decrease and the total information content obtained by the SAR imaging system will increase with an increase in the number of looks. Because the total average mutual information is also used to define a measure of radiometric resolution for radar images, it is shown that the radiometric resolution of a radar image of terrain will be improved by spatial averaging. In addition, the imaging process and the data compression process for SAR are each treated as an independent generalized communication channel. The effects of data compression upon radiometric resolution for SAR are studied and some conclusions are obtained.

Data Compression in Information Technology——High Speed Sampling Theorems

Shannon gave the sampling theorem about the band limited functions in 1948, but the Shannon's theorem cannot adapt to the need of modern high technology. This paper gives a new high speed sampling theorem which has a fast convergence rate, a high precision, and a simple algorithm. A practical example has been used to verify its efficiency.

Compression Design and Test of Water Vapor Receiving Data by Beidou Satellite

Water vapor monitoring system by Beidou satellite is a new detection system in meteorological department, which makes receiving amount of detected data and data storage and transmission pressure increase. Here, we try to use data compression to relieve pressure. Compres- sion software of water vapor monitoring system by Beidou satellite can be designed into three components： real-time compression software, check compression software and manual compression software, which respectively completes the compression tasks under real-time receiving, in-time check and separate compression, thereby forming a perfect compression system. Taking the design of manual compression software as guide,and using c language to develop,compression test of original receiving data is conducted. Test result proves that the system can carry out batch auto- matic compression, and compression rate can reach 30% ,which can reach the target of saving space in a degree.

The Compression Algorithm for the Data Acquisition System in HT-7 Tokamak

HT-7 superconducting tokamak in the Institute of Plasma Physics of the Chinese Academy of Sciences is an experimental device for fusion research in China. The main task of the data acquisition system of HT-7 is to acquire, store, analyze and index the data. The volume of the data is nearly up to hundreds of million bytes. Besides the hardware and software support, a great capacity of data storage, process and transfer is a more important problem. To deal with this problem, the key technology is data compression algorithm. In the paper, the data format in HT-7 is introduced first, then the data compression algorithm, LZO, being a kind of portable lossless data compression algorithm with ANSI C, is analyzed. This compression algorithm, which fits well with the data acquisition and distribution in the nuclear fusion experiment, offers a pretty fast compression and extremely fast decompression. At last the performance evaluation of LZO application in HT-7 is given.

A Ka-band Solid-state Transmitter Cloud Radar and Data Merging Algorithm for Its Measurements

This study concerns a Ka-band solid-state transmitter cloud radar, made in China, which can operate in three different work modes, with different pulse widths, and coherent and incoherent integration numbers, to meet the requirements for cloud remote sensing over the Tibetan Plateau. Specifically, the design of the three operational modes of the radar（i.e., boundary mode M1, cirrus mode M2, and precipitation mode M3） is introduced. Also, a cloud radar data merging algorithm for the three modes is proposed. Using one month＇s continuous measurements during summertime at Naqu on the Tibetan Plateau,we analyzed the consistency between the cloud radar measurements of the three modes. The number of occurrences of radar detections of hydrometeors and the percentage contributions of the different modes＇ data to the merged data were estimated.The performance of the merging algorithm was evaluated. The results indicated that the minimum detectable reflectivity for each mode was consistent with theoretical results. Merged data provided measurements with a minimum reflectivity of -35 dBZ at the height of 5 km, and obtained information above the height of 0.2 km. Measurements of radial velocity by the three operational modes agreed very well, and systematic errors in measurements of reflectivity were less than 2 dB. However,large discrepancies existed in the measurements of the linear depolarization ratio taken from the different operational modes.The percentage of radar detections of hydrometeors in mid- and high-level clouds increased by 60% through application of pulse compression techniques. In conclusion, the merged data are appropriate for cloud and precipitation studies over the Tibetan Plateau.

A real-time algorithm for broadband highdynamic seismic data compression

A new real-time algorithm of data compression, including the segment-normalized logical compression and socalled 'one taken from two samples',is presented for broadband high dynamic seismic recordings. This algorithm was tested by numerical simulation and data observed. Its results demonstrate that total errors in recovery data are less than 1% of original data in time domain,0.5% in frequency domain, when using these two methods together.Its compression ratio is greater than 3.The data compression softwares based on the algorithm have been used in the GDS-1000 portable broadband digital seismograph.

Satellite Data Reduction Using Entropy-preserved Image Compression Technique

In this paper, three techniques, line run coding, quadtree DF (Depth-First) representation and H coding for compressing classified satellite cloud images with no distortion are presented. In these three codings, the first two were invented by other persons and the third one, by ourselves. As a result, the comparison among their compression rates is. given at the end of this paper. Further application of these image compression technique to satellite data and other meteorological data looks promising.

System-on-Chip Test Data Compression Based on Split-Data Variable Length (SDV) Code

System-on-a-chips with intellectual property cores need a large volume of data for testing. The large volume of test data requires a large testing time and test data memory. Therefore new techniques are needed to optimize the test data volume, decrease the testing time, and conquer the ATE memory limitation for SOC designs. This paper presents a new compression method of testing for intellectual property core-based system-on-chip. The proposed method is based on new split- data variable length (SDV) codes that are designed using the split-options along with identification bits in a string of test data. This paper analyses the reduction of test data volume, testing time, run time, size of memory required in ATE and improvement of compression ratio. Experimental results for ISCAS 85 and ISCAS 89 Benchmark circuits show that SDV codes outperform other compression methods with the best compression ratio for test data compression. The decompression architecture for SDV codes is also presented for decoding the implementations of compressed bits. The proposed scheme shows that SDV codes are accessible to any of the variations in the input test data stream.

Image Representations of Numerical Simulations for Training Neural Networks

A large amount of data can partly assure good fitting quality for the trained neural networks.When the quantity of experimental or on-site monitoring data is commonly insufficient and the quality is difficult to control in engineering practice,numerical simulations can provide a large amount of controlled high quality data.Once the neural networks are trained by such data,they can be used for predicting the properties/responses of the engineering objects instantly,saving the further computing efforts of simulation tools.Correspondingly,a strategy for efficiently transferring the input and output data used and obtained in numerical simulations to neural networks is desirable for engineers and programmers.In this work,we proposed a simple image representation strategy of numerical simulations,where the input and output data are all represented by images.The temporal and spatial information is kept and the data are greatly compressed.In addition,the results are readable for not only computers but also human resources.Some examples are given,indicating the effectiveness of the proposed strategy.

Curvature Quantified Douglas-Peucker-based Phasor Measurement Unit Data Compression Method for Power System Situational Awareness

Facing constraints imposed by storage and bandwidth limitations,the vast volume of phasor meas-urement unit(PMU)data collected by the wide-area measurement system(WAMS)for power systems cannot be fully utilized.This limitation significantly hinders the effective deployment of situational awareness technologies for systematic applications.In this work,an effective curvature quantified Douglas-Peucker(CQDP)-based PMU data compression method is proposed for situational awareness of power systems.First,a curvature integrated distance(CID)for measuring the local flection and fluc-tuation of PMU signals is developed.The Doug-las-Peucker(DP)algorithm integrated with a quan-tile-based parameter adaptation scheme is then proposed to extract feature points for profiling the trends within the PMU signals.This allows adaptive adjustment of the al-gorithm parameters,so as to maintain the desired com-pression ratio and reconstruction accuracy as much as possible,irrespective of the power system dynamics.Fi-nally,case studies on the Western Electricity Coordinat-ing Council(WECC)179-bus system and the actual Guangdong power system are performed to verify the effectiveness of the proposed method.The simulation results show that the proposed method achieves stably higher compression ratio and reconstruction accuracy in both steady state and in transients of the power system,and alleviates the compression performance degradation problem faced by existing compression methods.Index Terms—Curvature quantified Douglas-Peucker,data compression,phasor measurement unit,power sys-tem situational awareness.

Efficient Multi-Resolution Compression Algorithm for Disk-Based Backup and Recovery

In this paper, we deal with the problem of improving backup and recovery performance by compressing redundancies in large disk-based backup system. We analyze some general compression algorithms; evaluate their scalability and applicability. We investigate the distribution features of the redundant data in whole system range, and propose a multi-resolution distributed compression algorithm which can discern duplicated data at granularity of file level, block level or byte level to reduce the redundancy in backup environment. In order to accelerate recovery, we propose a synthetic backup solution which stores data in a recovery-oriented way and can compose the final data in back-end backup server. Experiments show that this algorithm can greatly reduce bandwidth consumption, save storage cost, and shorten the backup and recovery time. We implement these technologies in our product, called H-info backup system, which is capable of achieving over 10x compression ratio in both network utilization and data storage during backup.

Distributed Radar Target Tracking with Low Communication Cost

In distributed radar,most of existing radar networks operate in the tracking fusion mode which combines radar target tracks for a higher positioning accuracy.However,as the filtering covariance matrix indicating positioning accuracy often occupies many bits,the communication cost from local sensors to the fusion is not always sufficiently low for some wireless communication chan-nels.This paper studies how to compress data for distributed tracking fusion algorithms.Based on the K-singular value decomposition(K-SVD)algorithm,a sparse coding algorithm is presented to sparsely represent the filtering covariance matrix.Then the least square quantization(LSQ)algo-rithm is used to quantize the data according to the statistical characteristics of the sparse coeffi-cients.Quantized results are then coded with an arithmetic coding method which can further com-press data.Numerical results indicate that this tracking data compression algorithm drops the com-munication bandwidth to 4%at the cost of a 16%root mean squared error(RMSE)loss.

Novel Image Encryption and Compression Scheme for IoT Environment

Latest advancements made in the processing abilities of smartdevices have resulted in the designing of Intelligent Internet of Things (IoT)environment. This advanced environment enables the nodes to connect, collect, perceive, and examine useful data from its surroundings. Wireless Multimedia Surveillance Networks (WMSNs) form a vital part in IoT-assistedenvironment since it contains visual sensors that examine the surroundingsfrom a number of overlapping views by capturing the images incessantly.Since IoT devices generate a massive quantity of digital media, it is thereforerequired to save the media, especially images, in a secure way. In order toachieve security, encryption techniques as well as compression techniques areemployed to reduce the amount of digital data, being communicated overthe network. Encryption Then Compression (ETC) techniques pave a wayfor secure and compact transmission of the available data to prevent unauthorized access. With this background, the current research paper presentsa new ETC technique to accomplish image security in IoT environment.The proposed model involves three major processes namely, IoT-based imageacquisition, encryption, and compression. The presented model involves optimal Signcryption Technique with Whale Optimization Algorithm (NMWOA)abbreviated as ST-NMWOA. The optimal key generation of signcryptiontechnique takes place with the help of NMWOA. Besides, the presented modelalso uses Discrete Fourier Transform (DFT) and Matrix Minimization (MM)algorithm-based compression technique. Extensive set of experimental analysis was conducted to validate the effective performance of the proposed model.The obtained values infer that the presented model is superior in terms of bothcompression efficiency and data secrecy in resource-limited IoT environment.

A Bit-level Text Compression Scheme Based on the ACW Algorithm

This paper presents a description and performance evaluation of a new bit-level, lossless, adaptive, and asymmetric data compression scheme that is based on the adaptive character wordlength （ACW（n）） algorithm. The proposed scheme enhances the compression ratio of the ACW（n） algorithm by dividing the binary sequence into a number of subsequences （s）, each of them satisfying the condition that the number of decimal values （d） of the n-bit length characters is equal to or less than 256. Therefore, the new scheme is referred to as ACW（n, s）, where n is the adaptive character wordlength and s is the number of subsequences. The new scheme was used to compress a number of text files from standard corpora. The obtained results demonstrate that the ACW（n, s） scheme achieves higher compression ratio than many widely used compression algorithms and it achieves a competitive performance compared to state-of-the-art compression tools.

Discovering Patterns in Symbolic Streams

The idea of discovering patterns in data seems to be essential for decision making in our social activities. This paper presents a simple algorithm which can be used to detect such patterns in elementary sequences of symbols. It adopts a data compression algorithm to seek for interesting patterns, which is important in understanding the way that the input symbol stream repeats itself In this paper, the algorithm and related data structures are described in detail, and an illustrative example is given.

An image compression method for space multispectral time delay and integration charge coupled device camera

Multispectral time delay and integration charge coupled device （TDICCD） image compression requires a low- complexity encoder because it is usually completed on board where the energy and memory are limited. The Consultative Committee for Space Data Systems （CCSDS） has proposed an image data compression （CCSDS-IDC） algorithm which is so far most widely implemented in hardware. However, it cannot reduce spectral redundancy in mukispectral images. In this paper, we propose a low-complexity improved CCSDS-IDC （ICCSDS-IDC）-based distributed source coding （DSC） scheme for multispectral TDICCD image consisting of a few bands. Our scheme is based on an ICCSDS-IDC approach that uses a bit plane extractor to parse the differences in the original image and its wavelet transformed coefficient. The output of bit plane extractor will be encoded by a first order entropy coder. Low-density parity-check-based Slepian-Wolf （SW） coder is adopted to implement the DSC strategy. Experimental results on space multispectral TDICCD images show that the proposed scheme significantly outperforms the CCSDS-IDC-based coder in each band.