An Efficient Crash Recovery Technique for Real-Time Main Memory Database

This paper presents an efficient recovery scheme suitable for real-time mainmemory database. In the recovery scheme, log records are stored in non-volatile RAM which is dividedinto four different partitions based on transaction types. Similarly, a main memory database isdivided into four partitions based data types. When the using ratio of log store area exceeds thethreshold value, checkpoint procedure is triggered. During executing checkpoint procedure, someuseless log records are deleted. During restart recovery after a crash, partition reloading policyis adopted to assure that critical data are reloaded and restored in advance, so that the databasesystem can be brought up before the entire database is reloaded into main memory. Therefore downtime is obvionsly reduced. Simulation experiments show our recovery scheme obviously improves thesystem performance, and does a favor to meet the dtadlints of real-time transactions.

Real-Time Memory Data Optimization Mechanism of Edge IoT Agent

With the full development of disk-resident databases(DRDB)in recent years,it is widely used in business and transactional applications.In long-term use,some problems of disk databases are gradually exposed.For applications with high real-time requirements,the performance of using disk database is not satisfactory.In the context of the booming development of the Internet of things,domestic real-time databases have also gradually developed.Still,most of them only support the storage,processing,and analysis of data values with fewer data types,which can not fully meet the current industrial process control system data types,complex sources,fast update speed,and other needs.Facing the business needs of efficient data collection and storage of the Internet of things,this paper optimizes the transaction processing efficiency and data storage performance of the memory database,constructs a lightweight real-time memory database transaction processing and data storage model,realizes a lightweight real-time memory database transaction processing and data storage model,and improves the reliability and efficiency of the database.Through simulation,we proved that the cache hit rate of the cache replacement algorithm proposed in this paper is higher than the traditional LRU(Least Recently Used)algorithm.Using the cache replacement algorithm proposed in this paper can improve the performance of the system cache.

Scheduling transactions in mobile distributed real-time database systems

A DMVOCC-MVDA （distributed multiversion optimistic concurrency control with multiversion dynamic adjustment） protocol was presented to process mobile distributed real-time transaction in mobile broadcast environments. At the mobile hosts, all transactions perform local pre-validation. The local pre-validation process is carried out against the committed transactions at the server in the last broadcast cycle. Transactions that survive in local pre-validation must be submitted to the server for local final validation. The new protocol eliminates conflicts between mobile read-only and mobile update transactions, and resolves data conflicts flexibly by using multiversion dynamic adjustment of serialization order to avoid unnecessary restarts of transactions. Mobile read-only transactions can be committed with no-blocking, and respond time of mobile read-only transactions is greatly shortened. The tolerance of mobile transactions of disconnections from the broadcast channel is increased. In global validation mobile distributed transactions have to do check to ensure distributed serializability in all participants. The simulation results show that the new concurrency control protocol proposed offers better performance than other protocols in terms of miss rate, restart rate, commit rate. Under high work load （think time is ls） the miss rate of DMVOCC-MVDA is only 14.6%, is significantly lower than that of other protocols. The restart rate of DMVOCC-MVDA is only 32.3%, showing that DMVOCC-MVDA can effectively reduce the restart rate of mobile transactions. And the commit rate of DMVOCC-MVDA is up to 61.2%, which is obviously higher than that of other protocols.

Validation concurrency control protocol in parallel real-time database systems

In parallel real-time database systems, concurrency control protocols must satisfy time constraints as well as the integrity constraints. The authors present a validation concurrency control(VCC) protocol, which can enhance the performance of real-time concurrency control mechanism by reducing the number of transactions that might miss their deadlines, and compare the performance of validation concurrency control protocol with that of HP2PL(High priority two phase locking) protocol and OCC-TI-WAIT-50(Optimistic concurrency control-time interval-wait-50) protocol under shared-disk architecture by simulation. The simulation results reveal that the protocol the author presented can effectively reduce the number of transactions restarting which might miss their deadlines and performs better than HP2PL and OCC-TI-WAIT-50. It works well when arrival rate of transaction is lesser than threshold. However, due to resource contention the percentage of missing deadline increases sharply when arrival rate is greater than the threshold.

A dynamic crash recovery scheme for distributed real-time database systems

Recovery performance in the event of failures is very important for distributed real-time database systems. This paper presents a time-cognizant logging-based crash recovery scheme （TCLCRS） that aims at distributed real-time databases, which adopts a main memory database as its ground support. In our scheme, each site maintains a real-time log for local transactions and the subtransactions, which execute at the site, and execte local checkpointing independently. Log records are stored in non-volatile high- speed store, which is divided into four different partitions based on transaction classes. During restart recovery after a site crash, partitioned crash recovery strategy is adopted to ensure that the site can be brought up before the entire local secondary database is reloaded in main memory. The partitioned crash recovery strategy not only guarantees the internal consistency to be recovered, but also guarantee the temporal consistency and recovery of the sates of physical world influenced by uncommitted transactions. Combined with two- phase commit protocol, TCLCRS can guarantee failure atomicity of distributed real-time transactions.

Quasi Serializable Concurrency Control in Distributed Real-Time Database Systems

This paper formally defines and analyses the new notion of correctness called quasi serializability, and then outlines corresponding concurrency control protocol QDHP for distributed real-time databases. Finally, through a series of simulation studies, it shows that using the new concurrency control protocol the performance of distributed real-time databases can be much improved.

Cache-Conscious Index Mechanism for Main-Memory Databases

Recent studies have addressed that the cache be havior is important in the design of main memory index structures. Cache-conscious indices such as the CSB^＋-tree are shown to outperform conventional main memory indices such as the AVL-tree and the T-tree. This paper proposes a cacheconscious version of the T-tree, CST-tree, defined according to the cache-conscious definition. To separate the keys within a node into two parts, the CST-tree can gain higher cache hit ratio.

A real-time prediction method for tunnel boring machine cutter-head torque using bidirectional long short-term memory networks optimized by multi-algorithm

Based on data from the Jilin Water Diversion Tunnels from the Songhua River(China),an improved and real-time prediction method optimized by multi-algorithm for tunnel boring machine(TBM)cutter-head torque is presented.Firstly,a function excluding invalid and abnormal data is established to distinguish TBM operating state,and a feature selection method based on the SelectKBest algorithm is proposed.Accordingly,ten features that are most closely related to the cutter-head torque are selected as input variables,which,in descending order of influence,include the sum of motor torque,cutter-head power,sum of motor power,sum of motor current,advance rate,cutter-head pressure,total thrust force,penetration rate,cutter-head rotational velocity,and field penetration index.Secondly,a real-time cutterhead torque prediction model’s structure is developed,based on the bidirectional long short-term memory(BLSTM)network integrating the dropout algorithm to prevent overfitting.Then,an algorithm to optimize hyperparameters of model based on Bayesian and cross-validation is proposed.Early stopping and checkpoint algorithms are integrated to optimize the training process.Finally,a BLSTMbased real-time cutter-head torque prediction model is developed,which fully utilizes the previous time-series tunneling information.The mean absolute percentage error(MAPE)of the model in the verification section is 7.3%,implying that the presented model is suitable for real-time cutter-head torque prediction.Furthermore,an incremental learning method based on the above base model is introduced to improve the adaptability of the model during the TBM tunneling.Comparison of the prediction performance between the base and incremental learning models in the same tunneling section shows that:(1)the MAPE of the predicted results of the BLSTM-based real-time cutter-head torque prediction model remains below 10%,and both the coefficient of determination(R^(2))and correlation coefficient(r)between measured and predicted values exceed 0.95;and(2)the incremental learning method is suitable for realtime cutter-head torque prediction and can effectively improve the prediction accuracy and generalization capacity of the model during the excavation process.

Domain semantic mapping of database metasearch engine

In order to implement semantic mapping of database metasearch engines, a system is proposed, which uses ontology as the organization form of information and records the new words not appearing in the ontology. When the new word＇ s frequency of use exceeds the threshold, it is added into the ontology. Ontology expansion is implemented in this way. The search process supports ＂and＂ and ＂or＂ Boolean operations accordingly. In order to improve the mapping speed of the system, a memory module is added which can memorize the recent query information of users and automatically learn the user＇ s query interest during the mapping which can dynamically decide the search order of instances tables. Experiments prove that these measures can obviously reduce the average mapping time.

Improvement Design for Distributed Real-Time Stream Processing Systems

In the era of Big Data, typical architecture of distributed real-time stream processing systems is the combination of Flume, Kafka, and Storm. As a kind of distributed message system, Kafka has the characteristics of horizontal scalability and high throughput, which is manly deployed in many areas in order to address the problem of speed mismatch between message producers and consumers. When using Kafka, we need to quickly receive data sent by producers. In addition, we need to send data to consumers quickly. Therefore, the performance of Kafka is of critical importance to the performance of the whole stream processing system. In this paper, we propose the improved design of real-time stream processing systems, and focus on improving the Kafka's data loading process.We use Kafka cat to transfer data from the source to Kafka topic directly, which can reduce the network transmission. We also utilize the memory file system to accelerate the process of data loading, which can address the bottleneck and performance problems caused by disk I/O. Extensive experiments are conducted to evaluate the performance, which show the superiority of our improved design.

A New Database Intrusion Detection Approach Based on Hybrid Meta-Heuristics

A new secured database management system architecture using intrusion detection systems(IDS)is proposed in this paper for organizations with no previous role mapping for users.A simple representation of Structured Query Language queries is proposed to easily permit the use of the worked clustering algorithm.A new clustering algorithm that uses a tube search with adaptive memory is applied to database log files to create users’profiles.Then,queries issued for each user are checked against the related user profile using a classifier to determine whether or not each query is malicious.The IDS will stop query execution or report the threat to the responsible person if the query is malicious.A simple classifier based on the Euclidean distance is used and the issued query is transformed to the proposed simple representation using a classifier,where the Euclidean distance between the centers and the profile’s issued query is calculated.A synthetic data set is used for our experimental evaluations.Normal user access behavior in relation to the database is modelled using the data set.The false negative(FN)and false positive(FP)rates are used to compare our proposed algorithm with other methods.The experimental results indicate that our proposed method results in very small FN and FP rates.

A sensor transaction scheduling algorithm for maintaining real-time data temporal validity

A new scheduling algorithm called deferrable scheduling with time slice exchange (DS-EXC) was proposed to maintain the temporal validity of real-time data. In DS-EXC, the time slice exchange method was designed to further defer the release time of transaction instances derived by the deferrable scheduling algorithm (DS-FP). In this way, more CPU time would be left for lower priority transactions and other transactions. In order to minimize the scheduling overhead, an off-line scheme was designed. In particular, the schedule for a transaction set is generated off-line until a repeating pattern is found, and then the pattern is used to construct the schedule on-line. The performance of DS-EXC was evaluated by sets of experiments. The results show that DS-EXC outperforms DS-FP in terms of increasing schedulable ratio. It also provides better performance under mixed workloads.

An Efficient Secure Real-Time Concurrency Control Protocol

Secure real-time databases must simultaneously satisfy two requirements in guaranteeing data security and minimizing the missing deadlines ratio of transactions. However, these two requirements can conflict with each other and achieve one requirement is to sacrifice the other. This paper presents a secure real-time concurrency control protocol based on optimistic method. The concurrency control protocol incorporates security constraints in a real-time optimistic concurrency control protocol and makes a suitable tradeoff between security and real-time requirements by introducing secure influence factor and real-time influence factor. The experimental results show the concurrency control protocol achieves data security without degrading real-time perform ance significantly.

Real-time Design Constraints in Implementing Active Vibration Control Algorithms

Although computer architectures incorporate fast processing hardware resources, high performance real-time implementation of a complex control algorithm requires an efficient design and software coding of the algorithm so as to exploit special features of the hardware and avoid associated architecture shortcomings. This paper presents an investigation into the analysis and design mechanisms that will lead to reduction in the execution time in implementing real-time control algorithms. The proposed mechanisms are exemplified by means of one algorithm, which demonstrates their applicability to real-time applications. An active vibration control （AVC） algorithm for a flexible beam system simulated using the finite difference （FD） method is considered to demonstrate the effectiveness of the proposed methods. A comparative performance evaluation of the proposed design mechanisms is presented and discussed through a set of experiments.

Research on Real-time Data Sharing in Power Integrated Automation System

The Power Integrated Automation System has a large amount of the real-time data, it needs to achieve data sharing in different modules in its own system, sometimes even needs to be shared with the other systems. The thesis discusses the characteristics and the ways of the real-time data sharing in the first place. Then, it compares the merits and drawbacks in different ways. Besides, it gives a viable resolution in different aspects, such as the design of the real-time database, the framework of the communication system and the design of the communication software, as well as the real-time data sharing in different systems.

A Real-time Data Model Based on Temporal Data

Real-time database systems contain not only transaction timing constraints, but also data timing constraints. This paper discusses the temporal characteristics of data in real-time databases and offers a definition of absolute and relative temporal consistency. In real-time database systems, it is often the case that the policies of transaction schedules only consider the deadline of real-time transactions, making it insufficient to ensure temporal correctness of transactions. A policy is given by considering both the deadlines of transactions and the “data deadline” to schedule real-time transactions. A real-time relational data model and a real-time relational algebra based on the characteristics of temporal data are also proposed. In this model, the temporal data has not only corresponding values, but also validity intervals corresponding to the data values. At the same time, this model is able to keep historical data values. When validity interval of a relation is [NOW, NOW], real-time relational algebra will transform to traditional relational algebra.

Research and Application of Synchronization Update of Distributed Heterogeneous Database

According to the different equipment, different system and heterogeneous database have be information ＂isolated island＂ problem, and the data of equipments can be updated in real time on the business node. The paper proposes a program of data synchronization platform based on J2EE （JMS） and XML, and detailed analysis and description of the workflow system, its frame structure and the key technology. Practice shows that this scheme has the advantages of convenient and real-time etc..

卷烟滤棒成型加工区域集控系统实现

针对滤棒成型作业区分散、设备种类多、制造厂家和数量多、接口协议复杂且不统一以及海量数据变量难识别等问题,设计并开发了基于OPC UA架构和实时内存数据库的就地滤棒加工区数采和监控系统。自主开发OPC服务器,实现了滤棒成型主、辅设备的实时数据采集,并通过高效内存数据库服务器对生产及设备信息进行海量存储,进而实现了对滤棒成型车间生产过程信息的全面、准确迁移集成,以及对滤棒生产、产量、质量、机台效率、设备状态等信息的可视化区域集控。

The Multi-level Recovery of Main-memory Real-time Database Systems with ECBH

Storing the whole database in the main-memory is a common method to processreal-time transaction in real-time database systems. The recovery mechanism of Main-memory Real-timeDatabase Systems (MMRTDBS) should reflect the characteristics of the main-memory database andreal-time database because their structures are quite different from other conventional databasesystems. In this paper, therefore, we propose a multi-level recovery mechanism for main-memoryreal-time database systems with Extendable Chained Bucket Hashing (ECBH). Owing to the occurrence ofreal-time data in real-time systems, we should also consider it in our recovery mechanism .According to our performance test, this mechanism can improve the transaction concurrency, reducingtransactions ' deadline missing rate.

MMOS:支持超卖的多租户数据库内存资源共享方法

多租户数据库为每个租户分配固定的资源配额,而这些资源配额通常未全部得到有效利用,这种静态分配策略导致资源利用率不高。若在不影响租户性能的前提下将未利用的空闲资源共享给其他租户使用,即实现资源超卖,则可以提高资源利用率、提升平台收益。为了支持资源超卖,需要准确预测租户的资源需求,动态地按需为租户分配资源。已有的针对多租户数据库的资源共享方法的研究对象主要是CPU资源,鲜有支持超卖的内存资源共享方法。鉴于此,在联机分析处理场景下,提出了一种支持超卖的多租户数据库内存资源共享方法MMOS(Multi-tenant database Memory resource Overselling and Sharing)。该方法通过准确预测每个租户的内存需求区间,按照区间上限为租户动态调整内存配额,在不影响租户性能的前提下,统一管理空闲内存资源以支持更多租户,实现内存超卖。实验结果表明,MMOS在租户负载动态变化的场景下具有较好效果。在不同资源量的资源池下,支持的租户数可以增加2~2.6倍,资源利用率峰值提升175%~238%。同时,每个租户的业务与性能未受影响。