Improvement Design for Distributed Real-Time Stream Processing Systems

In the era of Big Data, typical architecture of distributed real-time stream processing systems is the combination of Flume, Kafka, and Storm. As a kind of distributed message system, Kafka has the characteristics of horizontal scalability and high throughput, which is manly deployed in many areas in order to address the problem of speed mismatch between message producers and consumers. When using Kafka, we need to quickly receive data sent by producers. In addition, we need to send data to consumers quickly. Therefore, the performance of Kafka is of critical importance to the performance of the whole stream processing system. In this paper, we propose the improved design of real-time stream processing systems, and focus on improving the Kafka's data loading process.We use Kafka cat to transfer data from the source to Kafka topic directly, which can reduce the network transmission. We also utilize the memory file system to accelerate the process of data loading, which can address the bottleneck and performance problems caused by disk I/O. Extensive experiments are conducted to evaluate the performance, which show the superiority of our improved design.

A Flexible DSP-Based Network forReal-Time Image-Processing

This paper proposed a general purpose real-time image processing system based on a flexible DSP-based Network, which is implemented by a high bandwidth communication channel, links. The links is realized using FPGA and provides a bandwidth of 12. 8 Gbit/s. Using the links, The topologic of multi-DSP system can be changed online to meet the variabilities of the parallel algorithm of image processing. The system can be assembled with utmost tens of boards and maintain the high communication speed. Analysis of the system adaptivity to image processing is testified followed by actual results. Key words real-time image processing - multi-DSP - flexible - scalable - FPGA - links CLC number TP 303 Foundation item: Supported by the National Natural Science Foundation of China (60135020)Biography: MAO Hai-cen(1973-), male, Ph.D. candidate, research direction: artificial intelligence, expert system, pattern recognition and image processing

New multi-DSP parallel computing architecture for real-time image processing

The flexibility of traditional image processing system is limited because those system are designed for specific applications. In this paper, a new TMS320C64x-based multi-DSP parallel computing architecture is presented. It has many promising characteristics such as powerful computing capability, broad I/O bandwidth, topology flexibility, and expansibility. The parallel system performance is evaluated by practical experiment.

High Resolution Radar Real-Time Signal and Information Processing

Radar is an electronic device that uses radio waves to determine the range, angle, or velocity of objects. Real-time signal and information processor is an important module for real-time positioning, imaging, detection and recognition of targets. With the development of ultra-wideband technology, synthetic aperture technology, signal and information processing technology, the radar coverage, detection accuracy and resolution have been greatly improved, especially in terms of one-dimensional(1D) high-resolution radar detection, tracking, recognition, and two-dimensional(2D) synthetic aperture radar imaging technology. Meanwhile, for the application of radar detection and remote sensing with high resolution and wide swath, the amount of data has been greatly increased. Therefore, the radar is required to have low-latency and real-time processing capability under the constraints of size, weight and power consumption. This paper systematically introduces the new technology of high resolution radar and real-time signal and information processing. The key problems and solutions are discussed, including the detection and tracking of 1D high-resolution radar, the accurate signal modeling and wide-swath imaging for geosynchronous orbit synthetic aperture radar, and real-time signal and information processing architecture and efficient algorithms. Finally, the latest research progress and representative results are presented, and the development trends are prospected.

RFID Complex Event Processing: Applications in Real-Time Locating System

Complex event processing (CEP) can extract meaningful events for real-time locating system (RTLS) applications. To identify complex event accurately in RTLS, we propose a new RFID complex event processing method GEEP, which is based on the timed automata (TA) theory. By devising RFID locating application into complex events, we model the timing diagram of RFID data streams based on the TA. We optimize the constraint of the event streams and propose a novel method to derive the constraint between objects, as well as the constraint between object and location. Experiments prove the proposed method reduces the cost of RFID complex event processing, and improves the efficiency of the RTLS.

High performance reconfigurable hardware system for real-time image processing

A novel reconfigurable hardware system which uses both muhi-DSP and FPGA to attain high performance and real-time image processing are presented. The system structure and working principle of mainly processing multi-BSP board, extended multi-DSP board are analysed. The outstanding advantage is that the communication among different board components of this system is supported by high speed link ports ＆ serial ports for increasing the system performance and computational power. Then the implementation of embedded real-time operating systems （RTOS） by us is discussed in detail. In this system, we adopt two kinds of parallel structures controlled by RTOS for parallel processing of algorithms. The experimental results show that exploitive period of the system is short, and maintenance convenient. Thus it is suitable for real-time image processing and can get satisfactory effect of image recognition.

An FPGA-based real-time image processing system

This paper analyzes the current difficulties encountered in on-line inspection systems of strip surface quality, specifically relating to problems with real-time processing of huge amounts of data. To address this need, this paper describes an FPGA-based high-speed image processing module with both hardware and software aspects. Improving these two aspects together will help the system achieve real-time processing of massive image data, and simplifies the architecture of the strip surface quality on-line inspection system.

Low-Power Operational Amplifier for Real-Time Signal Processing System of Micro Air Vehicle

A low-power complementary metal oxide semiconductor（CMOS） operational amplifier （op-amp） for real-time signal processing of micro air vehicle （MAV） is designed in this paper.Traditional folded cascode architecture with positive channel metal oxide semiconductor（PMOS） differential input transistors and sub-threshold technology are applied under the low supply voltage.Simulation results show that this amplifier has significantly low power,while maintaining almost the same gain,bandwidth and other key performances.The power required is only 0.12 mW,which is applicable to low-power and low-voltage real-time signal acquisition and processing system.

Real-Time Discrete Adaptive Control of Robot Arm Based on Digital Signal Processing

A discrete model reference adaptive controller of robot arm is obtained by integrating the reduced dynamic model of robot, model reference adaptive control （MRAC） and digital signal processing （DSP） computer system into an electromechanical system. With the DSP computer system, the control signal of each joint of the robot arm can be processed in real time and independently. The simulation and experiment results show that with the control strategy, the robot achieved a good trajectory following precision, a good decoupling performance and a high real-time adaptivity.

Intercross real-time control strategy in alternating activated sludge process for short-cut biological nitrogen removal treating domestic wastewater

To develop technically feasible and economically favorable dynamic process control(DPC)strategies for an alternating activated sludge(AAS)system,a bench-scale continuous-flow alternating aerobic and anoxic reactor,performing short-cut nitrogen removal from real domestic wastewater was operated under different control strategies for more than five months.A fixed-time control(FTC) study showed that bending-points on pH and oxidation-reduction potential(ORP)profiles accurately coincided with the major biologic...

Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Heterogeneity induced strain localization in block-in-matrix-soils subjected to uniaxial loading using real-time CT scanning

Block-in-matrix-soils(bimsoils)are geological mixtures that have distinct structures consisting of relatively strong rock blocks and weak matrix soils.It is still a challenge to evaluate the mechanical behaviors of bimsoils because of the heterogeneity,chaotic structure,and lithological variability.As a result,only very limited laboratory studies have been reported on the evolution of their internal deformation.In this study,the deformation evolution of bimsoils under uniaxial loading is investigated using real-time X-ray computed tomography(CT)and image correlation algorithm(with a rock block percentage(RBP)of 40%).Three parameters,i.e.heterogeneity coefficient(K),correlation coefficient(CC),and standard deviation(STD)of displacement fields,are proposed to quantify the heterogeneity of the motion of the rock blocks and the progressive deformation of the bimsoils.Experimental results show that the rock blocks in bimsoils are prone to forming clusters with increasing loading,and the sliding surface goes around only one side of a cluster.Based on the movement of the rock blocks recorded by STD and CC,the progressive deformation of the bimsoils is quantitatively divided into three stages:initialization of the rotation of rock blocks,formation of rock block clusters,and formation of a shear band by rock blocks with significant rotation.Moreover,the experimental results demonstrate that the meso-motion of rock blocks controls the macroscopic mechanical properties of the samples.

Real-Time Memory Data Optimization Mechanism of Edge IoT Agent

With the full development of disk-resident databases(DRDB)in recent years,it is widely used in business and transactional applications.In long-term use,some problems of disk databases are gradually exposed.For applications with high real-time requirements,the performance of using disk database is not satisfactory.In the context of the booming development of the Internet of things,domestic real-time databases have also gradually developed.Still,most of them only support the storage,processing,and analysis of data values with fewer data types,which can not fully meet the current industrial process control system data types,complex sources,fast update speed,and other needs.Facing the business needs of efficient data collection and storage of the Internet of things,this paper optimizes the transaction processing efficiency and data storage performance of the memory database,constructs a lightweight real-time memory database transaction processing and data storage model,realizes a lightweight real-time memory database transaction processing and data storage model,and improves the reliability and efficiency of the database.Through simulation,we proved that the cache hit rate of the cache replacement algorithm proposed in this paper is higher than the traditional LRU(Least Recently Used)algorithm.Using the cache replacement algorithm proposed in this paper can improve the performance of the system cache.

Real-time monitoring of weld penetration quality in robotic arc welding process

It is of great significance to develop an intelligent monitoring system for weld penetration defects such as incomplete penetration and burn-through in real-time during robotic arc welding process. In this paper, robotic gas metal arc welding experiments are carried out on the mild steel test pieces with Vee-type groove. Through-the-arc sensing method is used to capture the transient values of the welding voltage and current. The raw data of the captured welding current and voltage are processed statistically, and the feature vector SIO is extracted to correlate the welding conditions to the weld penetration information. It lays foundation for intelligent monitoring of weld quality in robotic arc welding.

Comparison of Two Real-time PCR Technigues for Quantification of GMO Contents in Highly Processed Products of Soybean

The RR soybean was quantitatively detected by ABI Prism 7300 sequence detector with PCR primers and fluorescence probes were designed according to the sequences of endogenous Lectin gene and exogenous CP4-EPSPS gene, and the PCR systems were based on SYBR Green I and TaqMan. The standard curve of ACt between CP4-EPSPS gene and Lectin gene of the RR soybean in standard materials was generated and a linear regression equation was obtained. Quantification methods were optimized through two different real-time PCR chemistries, i.e. SYBR Green I and TaqMan, and the RR soybean contents were quantified in five standard samples and seven highly processed products by the two assays. Both methods are proved to be specific, highly sensitive and reliable for both identification and quantification of soybean DNA. The results indicate that the two optimized PCR system can be used for the practical quantitative detection of RR soybean in highly processed products.

Label-Free and Real-Time Monitor of Binding and Dissociation Processes between Protein A and Swine IgG by Oblique-Incidence Reflectivity Difference Method

Life science has a need for detection methods that are label-free and real-time. In this paper, we have selected staphylococcal protein A （SPA） and swine immunoglobulin G （IgG）, and monitor the bindings between SPA and swine IgG with different concentrations, as well as the dissociations of SPA-swine IgG complex in different pH values of phosphate buffer by oblique-incidence reflectivity difference （OIRD） in a label-free and real-time fashion. We obtain the ON and OFF reaction dynamic curves corresponding to the bindings and dissociations of SPA and swine IgG. Through our analysis of the experimental results, we have been able to obtain the damping coefficients and the dissociation time of SPA and swine IgG for different pH values of the phosphate buffer. The results prove that the OIRD technique is a competing method for monitoring the dynamic processes of biomolecule interaction and achieving the quantitative information of reaction kinetics.

Joint Algorithm of Message Fragmentation and No-Wait Scheduling for Time-Sensitive Networks

Time-sensitive networks(TSNs)support not only traditional best-effort communications but also deterministic communications,which send each packet at a deterministic time so that the data transmissions of networked control systems can be precisely scheduled to guarantee hard real-time constraints.No-wait scheduling is suitable for such TSNs and generates the schedules of deterministic communications with the minimal network resources so that all of the remaining resources can be used to improve the throughput of best-effort communications.However,due to inappropriate message fragmentation,the realtime performance of no-wait scheduling algorithms is reduced.Therefore,in this paper,joint algorithms of message fragmentation and no-wait scheduling are proposed.First,a specification for the joint problem based on optimization modulo theories is proposed so that off-the-shelf solvers can be used to find optimal solutions.Second,to improve the scalability of our algorithm,the worst-case delay of messages is analyzed,and then,based on the analysis,a heuristic algorithm is proposed to construct low-delay schedules.Finally,we conduct extensive test cases to evaluate our proposed algorithms.The evaluation results indicate that,compared to existing algorithms,the proposed joint algorithm improves schedulability by up to 50%.

Real-Time Hybrid Simulation of Seismically Isolated Structures with Full-Scale Bearings and Large Computational Models

Hybrid simulation can be a cost effective approach for dynamic testing of structural components at full scale while capturing the system level response through interactions with a numerical model.The dynamic response of a seismically isolated structure depends on the combined characteristics of the ground motion,bearings,and superstructure.Therefore,dynamic full-scale system level tests of isolated structures under realistic dynamic loading conditions are desirable towards a holistic validation of this earthquake protection strategy.Moreover,bearing properties and their ultimate behavior have been shown to be highly dependent on rate-of-loading and scale size effects,especially under extreme loading conditions.Few laboratory facilities can test full-scale seismic isolation bearings under prescribed displacement and/or loading protocols.The adaptation of a full-scale bearing test machine for the implementation of real-time hybrid simulation is presented here with a focus on the challenges encountered in attaining reliable simulation results for large scale dynamic tests.These advanced real-time hybrid simulations of large and complex hybrid models with several thousands of degrees of freedom are some of the first to use high performance parallel computing to rapidly execute the numerical analyses.Challenges in the experimental setup included measured forces contaminated by delay and other systematic control errors in applying desired displacements.Friction and inertial forces generated by the large-scale loading apparatus can affect the accuracy of measured force feedbacks.Reliable results from real-time hybrid simulation requires implementation of compensation algorithms and correction of these various sources of errors.Overall,this research program confirms that real-time hybrid simulation is a viable testing method to experimentally assess the behavior of full-scale isolators while capturing interactions with the numerical models of the superstructure to evaluate system level and in-structure response.

Applications of advanced signal processing and machine learning in the neonatal hypoxic-ischemic electroencephalography

Perinatal hypoxic-ischemic-encephalopathy significantly contributes to neonatal death and life-long disability such as cerebral palsy. Advances in signal processing and machine learning have provided the research community with an opportunity to develop automated real-time identification techniques to detect the signs of hypoxic-ischemic-encephalopathy in larger electroencephalography/amplitude-integrated electroencephalography data sets more easily. This review details the recent achievements, performed by a number of prominent research groups across the world, in the automatic identification and classification of hypoxic-ischemic epileptiform neonatal seizures using advanced signal processing and machine learning techniques. This review also addresses the clinical challenges that current automated techniques face in order to be fully utilized by clinicians, and highlights the importance of upgrading the current clinical bedside sampling frequencies to higher sampling rates in order to provide better hypoxic-ischemic biomarker detection frameworks. Additionally, the article highlights that current clinical automated epileptiform detection strategies for human neonates have been only concerned with seizure detection after the therapeutic latent phase of injury. Whereas recent animal studies have demonstrated that the latent phase of opportunity is critically important for early diagnosis of hypoxic-ischemic-encephalopathy electroencephalography biomarkers and although difficult, detection strategies could utilize biomarkers in the latent phase to also predict the onset of future seizures.

Confidentiality-aware message scheduling for security-critical wireless networks

Without considering security, existing message scheduling mechanisms may expose critical messages to malicious threats like confidentiality attacks. Incorporating confidentiality improvement into message scheduling, this paper investigates the problem of scheduling aperiodc messages with time-critical and security-critical requirements. A risk-based security profit model is built to quantify the security quality of messages; and a dynamic programming based approximation algorithm is proposed to schedule aperiodic messages with guaranteed security performance. Experimental results illustrate the efficiency and effectiveness of the proposed algorithm.