Key-Value Store Coupled with an Operating System for Storing Large-Scale Values

The key-value store can provide flexibility of data types because it does not need to specify the data types to be stored in advance and can store any types of data as the value of the key-value pair.Various types of studies have been conducted to improve the performance of the key-value store while maintaining its flexibility.However,the research efforts storing the large-scale values such as multimedia data files(e.g.,images or videos)in the key-value store were limited.In this study,we propose a new key-value store,WR-Store++aiming to store the large-scale values stably.Specifically,it provides a new design of separating data and index by working with the built-in data structure of the Windows operating system and the file system.The utilization of the built-in data structure of the Windows operating system achieves the efficiency of the key-value store and that of the file system extends the limited space of the storage significantly.We also present chunk-based memory management and parallel processing of WR-Store++to further improve its performance in the GET operation.Through the experiments,we show that WR-Store++can store at least 32.74 times larger datasets than the existing baseline key-value store,WR-Store,which has the limitation in storing large-scale data sets.Furthermore,in terms of processing efficiency,we show that WR-Store++outperforms not only WR-Store but also the other state-ofthe-art key-value stores,LevelDB,RocksDB,and BerkeleyDB,for individual key-value operations and mixed workloads.

PRS:Predication-Based Replica Selection Algorithm for Key-Value Stores

The tail latency of end-user requests,which directly impacts the user experience and the revenue,is highly related to its corresponding numerous accesses in key-value stores.The replica selection algorithm is crucial to cut the tail latency of these key-value accesses.Recently,the C3 algorithm,which creatively piggybacks the queue-size of waiting keys from replica servers for the replica selection at clients,is proposed in NSDI 2015.Although C3 improves the tail latency a lot,it suffers from the timeliness issue on the feedback information,which directly influences the replica selection.In this paper,we analysis the evaluation of queuesize of waiting keys of C3,and some findings of queue-size variation were made.It motivate us to propose the Prediction-Based Replica Selection(PRS)algorithm,which predicts the queue-size at replica servers under the poor timeliness condition,instead of utilizing the exponentially weighted moving average of the state piggybacked queue-size as in C3.Consequently,PRS can obtain more accurate queue-size at clients than C3,and thus outperforms C3 in terms of cutting the tail latency.Simulation results confirm the advantage of PRS over C3.

分布式key-value系统错误污染检测

随着key-value存储系统的广泛使用,越来越多的研究开始关注分布式系统中的可信问题,其中一个重要的问题是,如何在系统被入侵或者管理员配置错误并运行一段时间后,检测出受污染的数据,从而可以在恢复错误数据的同时保留系统的合法更新.文中提出了一种基于key-value存储系统的错误污染检测方法,该方法允许在客户端不可信的前提下,检测客户端之间的污染扩散.文中设计了一种基于各服务器逻辑时钟的向量时钟,该时钟以用户关联操作为更新规则,记录了跨服务器的读写请求逻辑关系,用以进行错误污染跟踪;同时为了减少大规模系统中污染检测的开销,基于该向量时钟,文中进一步提出了一种在分布式系统中由操作序列构成有向无环图的污染分析方法.基于广泛使用的Voldmort key-value系统,文中实现了一个错误污染检测系统,Tracker Store.在集群测试环境下,文中对引入新的检测系统后产生的额外延迟开销进行了测试.

A Distributed In-Memory Database Solution for Mass Data Applications

In this paper, a Distributed In-Memory Database （DIMDB） system is proposed to improve processing efficiency in mass data applications. The system uses an enhanced language similar to Structured Query Language （SQL） with a key-value storage schema. The design goals of the DIMDB system is described and its system architecture is discussed. Operation flow and the enhanced SOL-like language are also discussed, and experimental results are used to test the validity of the system.

应用App Store模式 建设电信企业级应用系统

为了实现信令监测系统"灵活数据采集、统一数据共享、应用快速复制"的目标,避免"烟囱式"信令应用的出现,本文提出利用目前个人应用市场中风行的App Store模式来构建信令监测系统。本文对"烟囱式"系统产生的根源进行了分析,并进一步对采用App Store模式建设企业级应用所具有的优势、实施的关键点和App Store架构与发布共享流程进行了初步的探讨和论证。

Design and Implementation of Key-Value Database for Ship Virtual Test Platform Based on Distributed System

The virtual test platform is a vital tool for ship simulation and testing.However,the numerical pool ship virtual test platform is a complex system that comprises multiple heterogeneous data types,such as relational data,files,text,images,and animations.The analysis,evaluation,and decision-making processes heavily depend on data,which continue to increase in size and complexity.As a result,there is an increasing need for a distributed database system to manage these data.In this paper,we propose a Key-Value database based on a distributed system that can operate on any type of data,regardless of its size or type.This database architecture supports class column storage and load balancing and optimizes the efficiency of I/O bandwidth and CPU resource utilization.Moreover,it is specif-ically designed to handle the storage and access of largefiles.Additionally,we propose a multimodal data fusion mechanism that can connect various descrip-tions of the same substance,enabling the fusion and retrieval of heterogeneous multimodal data to facilitate data analysis.Our approach focuses on indexing and storage,and we compare our solution with Redis,MongoDB,and MySQL through experiments.We demonstrate the performance,scalability,and reliability of our proposed database system while also analysing its architecture’s defects and providing optimization solutions and future research directions.In conclu-sion,our database system provides an efficient and reliable solution for the data management of the virtual test platform of numerical pool ships.

基于MQTT协议的轻量化文本信息分发技术研究

随着卫星通信网络的发展,基于卫星通信网络实现手机、平板电脑等移动通信设备之间的通信成为研究的热点。由于其采用无线电波进行信号传输,且需要经过多个中继站进行信号的转发和处理,导致存在网络延时大、丢包率高以及信道狭窄等问题。针对文本信息在卫星通信网络下的分发过程存在效率低下、可靠性不高的问题,设计一种基于MQTT(Message Queuing Telemetry Transport)协议的轻量化文本信息分发技术。该技术使用MQTT协议作为消息传输协议,在文本信息分发前对MQTT协议进行主题设计、发布订阅机制设计、设备连接设计以及设备心跳设计,确保设备之间的连通性;在文本信息的分发过程中,设计数据校验加密算法、文本信息轻量化处理方法和离线消息存储机制,保证文本信息分发的安全性、可靠性。实验结果表明:相较于传统的基于JSON数据的文本信息分发技术,该技术在提高文本信息分发效率的同时,可确保信息分发的安全性、完整性和稳定性。

Hybrid Content Distribution Framework for Large-Scale Vehicular Ad Hoc Networks

Content distribution in large-scale vehicular ad hoc networks is difficult due to the scalability issue. A message may need to be carried by several vehicles till it reaches the destination. To select an appropriate next-hop carrier, the current carrier should ex- change control messages with a large number of vehicles encountered, and thus the pure ad hoc solution is not scalable. In this paper, we introduce a hybrid-network solution. We first divide the area into regions, and select a hot spot in each region to install a road-side unit （RSU）. RSUs can coordinate message exchanges between vehicles, and storage devices are used to temporarily hold a message waiting for the next-hop carrier. The RSUs and the vehicles traveling between them construct an overlay store-car- ry-and-forward content distribution network. Two types of vehicles exist, one with fixed mobility patterns such as buses, and the other with random patterns such as taxis. Considering one or both types of vehicles, utility-based optimization problems can be formulated to find the optimal routing solutions. Using the bus and taxi traces of Shanghai city, we demonstrate the effectiveness of the hybrid framework in terms of delivery delay, delivery ratio and overhead ratio.

基于ISSA-LSTM的储麦长期品质预测

为了解决非时序预测模型无法预测储麦品质时序劣变趋势,以及现有数据驱动的时序预测模型在长期储麦品质预测中因样本不足导致长期预测精度不高的问题,提出一种基于改进麻雀搜索算法(ISSA)优化长短时记忆网络(LSTM)的长期储麦品质预测模型。首先,提出了一种统计均匀分布方法,利用小麦稳定劣化的生理知识对原始数据进行增强扩容。其次,利用麻雀搜索算法(SSA)对LSTM模型进行优化,克服局部极值点,提高收敛速度。最后,引入t分布函数对SSA位置更新过程进行扰动避免局部最优。结果表明,储麦品质参数中的吸水率、咀嚼度、脂肪酸值和峰值黏度与储藏时间的Spearman相关性较为显著,相关系数均高于0.9,ISSA-LSTM模型预测精度相比于BP、LSTM、SSA-LSTM预测模型分别提高了11.83%、16.98%、26.50%,有助于提高小麦品质预测及分析的准确性。

外挂物气动特性的数值模拟和风洞试验

根据外挂物高、低速风洞测压试验数据和CFD数值仿真模拟,获得了左右侧机翼下外挂物的气动特性。基于“三层次”网格生成技术,利用有限体积法求解带有湍流模型的Navier-Stokes方程,对外挂物扰流流场进行数值模拟,结合高、低速测压风洞试验,验证了研究手段的有效性,分析了不同气动力系数随攻角、侧滑角及马赫数的变化情况。结果表明:高、低速下外挂物升力系数受攻角影响的规律一致;在正侧滑时,机身的遮挡对右机翼下外挂物的侧向力系数和偏航力矩系数的影响较大。

实用化数据同步技术在新型配电系统规划多人协同机制中的应用

随着能源互联网时代的到来,将各能源部门内部或部门之间使用的不同数据库整合成统一的数据平台,既适应能源互联网发展趋势,又能为新型配电系统网格化规划多人协同机制的实现奠定基础。异构数据库的同步,关键在于源端数据变更的捕获、目的端数据操作的辨识以及目的端数据的同步。在源端变更数据捕获方面,通过创建临时表获取源端的变更数据,大大减少了传输数据量。在数据操作辨识方面,采用源端利用组合编码统一标识更新数据、目的端自行解析的方式,避免了解读的混乱。为省去大量同类语句的解析时间,在目的端事先制定好各类数据操作的存储过程,以实现同步数据的便捷写入。将该实用化数据同步技术应用在株洲市新型配电系统能源互联网规划多人协同机制中,成功提高了效率,证明了方法的有效性和可行性。实用化数据同步技术既能将源端的变更数据以系统开销最小的方式准确抓取出来,又能使目的端的数据写入更加快捷。

RS-store:RDMA-enabled skiplist-based key-value store for efficient range query

Many key-value stores use RDMA to optimize the messaging and data transmission between application layer and the storage layer,most of which only provide point-wise operations.Skiplist-based store can support both point operations and range queries,but its CPU-intensive access operations combined with the high-speed network will easily lead to the storage layer reaches CPU bottlenecks.The common solution to this problem is offloading some operations into the application layer and using RDMA bypassing CPU to directly perform remote access,but this method is only used in the hash tablebased store.In this paper,we present RS-store,a skiplist-based key-value store with RDMA,which can overcome the CPU handle of the storage layer by enabling two access modes:local access and remote access.In RS-store,we redesign a novel data structure R-skiplist to save the communication cost in remote access,and implement a latch-free concurrency control mechanism to ensure all the concurrency during two access modes.RS-store also supports client-active range query which can reduce the storage layer’s CPU consumption.At last,we evaluate RS-store on an RDMA-capable cluster.Experimental results show that RS-store achieves up to 2x improvements over RDMA-enabled RocksDB on the throughput and application’s scalability.

R-Memcached: A Reliable In-Memory Cache for Big Key-Value Stores

Large-scale key-value stores are widely used in many Web-based systems to store huge amount of data as(key, value) pairs. In order to reduce the latency of accessing such(key, value) pairs, an in-memory cache system is usually deployed between the front-end Web system and the back-end database system. In practice, a cache system may consist of a number of server nodes, and fault tolerance is a critical feature to maintain the latency Service-Level Agreements(SLAs). In this paper, we present the design, implementation, analysis, and evaluation of R-Memcached, a reliable in-memory key-value cache system that is built on top of the popular Memcached software. R-Memcached exploits coding techniques to achieve reliability, and can tolerate up to two node failures.Our experimental results show that R-Memcached can maintain very good latency and throughput performance even during the period of node failures.

太阳能跨季节蓄热和短期蓄热供热系统应用对比研究

将集中式系统小型化,尝试设计分布式供热系统。采用太阳能跨季节蓄热和太阳能短期蓄热分别对张家口市某一单体农户住宅进行采暖设计。对于两种蓄热方式分别选取合适的集热器面积,确定合适的蓄热水池体积。通过计算对比采用两种蓄热方式采暖的实用性与经济性。

MyWAL:performance optimization by removing redundant input/output stack in key-value store

Based on a log-structured merge(LSM)tree,the key-value(KV)storage system can provide high reading performance and optimize random writing performance.It is widely used in modern data storage systems like e-commerce,online analytics,and real-time communication.An LSM tree stores new KV data in the memory and flushes to disk in batches.To prevent data loss in memory if there is an unexpected crash,RocksDB appends updating data in the write-ahead log(WAL)before updating the memory.However,synchronous WAL significantly reduces writing performance.In this paper,we present a new WAL mechanism named MyWAL.It directly manages raw devices(or partitions)instead of saving data on a traditional file system.These can avoid useless metadata updating and write data sequentially on disks.Experimental results show that MyWAL can significantly improve the data writing performance of RocksDB compared to the traditional WAL for small KV data on solid-state disks(SSDs),as much as five to eight times faster.On non-volatile memory express soild-state drives(NVMe SSDs)and non-volatile memory(NVM),MyWAL can improve data writing performance by 10%–30%.Furthermore,the results of YCSB(Yahoo!Cloud Serving Benchmark)show that the latency decreased by 50%compared with SpanDB.

基于改进灰狼算法的含分布式电源配电网无功优化

为提高含分布式电源配电网电压质量和减少线路损耗,建立以配电网网络损耗最小和电压偏移最小为目标的无功优化模型,模型包含光伏发电、蓄电池储能和配电网线路潮流等设备运行约束条件;针对模型的非线性,采用改进灰狼优化算法对模型进行求解。利用改进的IEEE33节点配电系统进行仿真验证,结果表明经过优化后配电系统电压质量得到改善且网络损耗有所减少,另外,与改进前灰狼算法和帝国主义竞争算法(imperialist competition algorithm,ICA)计算结果对比,改进的灰狼算法提高了其全局寻优能力,验证了模型和算法的有效性。

新型电网-微电网(Microgrid)研究综述

首先介绍了微电网的产生背景,并对微电网进行定义。其次,在对比了各国微电网的研究进展现状后,针对一个典型的微电网阐述了基本运行方式。接着介绍了微电网的电源和储能形式,并对微源的控制器特性即频率下垂曲线进行了简单的描述。微电网输配电系统也与传统电网有着明显的不同,由于低压线路参数的特殊性,需要对功率调节公式进行修正。微电网系统优化与稳定也是微电网的关键研究内容之一,初步给出了系统优化和稳定运行目标。最后,在依托电工所在新能源方面的既有成果上,提出了微电网的将来研究的重点和难点。

三维流体力学预测风机不同布置形式对冷库气流的影响

我国冷库的建设数量近年来增长迅速,可多数冷库中的气流分布并不合理,而风机的布置形式是影响冷库中气流分布的一个重要因素。运用SIMPLE算法,结合波兴涅克(Boussineq)假设,对目标冷库中不同的冷风机安装形式进行三维数值模拟。模拟结果显示目标冷库中采用两个三出风口的风机吹风比采用三个两出风口的风机吹风所引起的气流场均匀性更好;同时针对后者造成的射流偏移,对风机位置进行改变后得出了冷库中的流场较改变前均匀性提高。数值模拟的结果对中型冷库和大型冷库中风机的安装形式和安装位置提供了参考依据。

有限元与Monte Carlo方法耦合的冷轧纯铝板再结晶模拟

利用有限元软件ANSYS模拟的冷轧铝板应力场及其相应的储能场,在假设的介观非均匀储能场基础上,考虑冷轧铝板的宏观能量场的非均匀性,实现了金属变形的有限元方法与Monte Carlo再结晶模拟方法的耦合,有效地模拟非均匀储能场基础上的冷轧铝板再结晶过程,结合现有的理论与实验结果,比较和验证了两个极限储能部位的介观结构拓扑组织和再结晶动力学参数,得到较为理想的结果。

城市大型连锁超市的空间布局模式探析——以广州“好又多”连锁超市为例

通过对广州大型连锁超市“好又多”空间布局影响因素的分析,系统的检验大型连锁超市在城市不同区域的的空间分布规律。分析认为,大型超市在城市的分布基本上是比较均匀的,一般会在主干道上或者交叉路口并偏向于居住区附近选址;超市的规模跟人流量是成较强的正相关关系,但是并不是取决于该区人口的多少;同时,区位、交通量、土地利用方式、人口数量及密度、人均工资、区域面积等都影响着超市的空间布局与空间拓展。