基于Linux内核的Key-Value存储系统——KStore

Key-Value存储系统在各种互联网服务中被广泛使用,但现有的Key-Value存储系统通常在用户态空间设计和实现,因为频繁的模式切换和上下文切换,导致访问接口、事务处理效率不高,在高并发、低延迟的数据存储需求中尤为突出。针对该问题,给出了一个内核态Key-Value存储系统的实现——KStore:提供内核空间的索引和内存分配机制,并在此基础上,通过基于内核Socket的远程接口以及基于文件系统的本地接口,保证了KStore的低延迟;同时,通过基于内核多线程的并发处理机制,保证了KStore的并发性。实验结果表明,与Memcached相比,KStore在实时性和并发性方面都取得显著优势。

KSKV:Key-Strategy for Key-Value Data Collection with Local Differential Privacy

In recent years,the research field of data collection under local differential privacy(LDP)has expanded its focus fromelementary data types to includemore complex structural data,such as set-value and graph data.However,our comprehensive review of existing literature reveals that there needs to be more studies that engage with key-value data collection.Such studies would simultaneously collect the frequencies of keys and the mean of values associated with each key.Additionally,the allocation of the privacy budget between the frequencies of keys and the means of values for each key does not yield an optimal utility tradeoff.Recognizing the importance of obtaining accurate key frequencies and mean estimations for key-value data collection,this paper presents a novel framework:the Key-Strategy Framework forKey-ValueDataCollection under LDP.Initially,theKey-StrategyUnary Encoding(KS-UE)strategy is proposed within non-interactive frameworks for the purpose of privacy budget allocation to achieve precise key frequencies;subsequently,the Key-Strategy Generalized Randomized Response(KS-GRR)strategy is introduced for interactive frameworks to enhance the efficiency of collecting frequent keys through group-anditeration methods.Both strategies are adapted for scenarios in which users possess either a single or multiple key-value pairs.Theoretically,we demonstrate that the variance of KS-UE is lower than that of existing methods.These claims are substantiated through extensive experimental evaluation on real-world datasets,confirming the effectiveness and efficiency of the KS-UE and KS-GRR strategies.

Key-Value Store Coupled with an Operating System for Storing Large-Scale Values

The key-value store can provide flexibility of data types because it does not need to specify the data types to be stored in advance and can store any types of data as the value of the key-value pair.Various types of studies have been conducted to improve the performance of the key-value store while maintaining its flexibility.However,the research efforts storing the large-scale values such as multimedia data files(e.g.,images or videos)in the key-value store were limited.In this study,we propose a new key-value store,WR-Store++aiming to store the large-scale values stably.Specifically,it provides a new design of separating data and index by working with the built-in data structure of the Windows operating system and the file system.The utilization of the built-in data structure of the Windows operating system achieves the efficiency of the key-value store and that of the file system extends the limited space of the storage significantly.We also present chunk-based memory management and parallel processing of WR-Store++to further improve its performance in the GET operation.Through the experiments,we show that WR-Store++can store at least 32.74 times larger datasets than the existing baseline key-value store,WR-Store,which has the limitation in storing large-scale data sets.Furthermore,in terms of processing efficiency,we show that WR-Store++outperforms not only WR-Store but also the other state-ofthe-art key-value stores,LevelDB,RocksDB,and BerkeleyDB,for individual key-value operations and mixed workloads.

PRS:Predication-Based Replica Selection Algorithm for Key-Value Stores

The tail latency of end-user requests,which directly impacts the user experience and the revenue,is highly related to its corresponding numerous accesses in key-value stores.The replica selection algorithm is crucial to cut the tail latency of these key-value accesses.Recently,the C3 algorithm,which creatively piggybacks the queue-size of waiting keys from replica servers for the replica selection at clients,is proposed in NSDI 2015.Although C3 improves the tail latency a lot,it suffers from the timeliness issue on the feedback information,which directly influences the replica selection.In this paper,we analysis the evaluation of queuesize of waiting keys of C3,and some findings of queue-size variation were made.It motivate us to propose the Prediction-Based Replica Selection(PRS)algorithm,which predicts the queue-size at replica servers under the poor timeliness condition,instead of utilizing the exponentially weighted moving average of the state piggybacked queue-size as in C3.Consequently,PRS can obtain more accurate queue-size at clients than C3,and thus outperforms C3 in terms of cutting the tail latency.Simulation results confirm the advantage of PRS over C3.

Key-Value数据库的研究与应用

首先介绍Key-Value数据库的概念,接着比较了Key-Value数据库与关系数据库的区别和优劣点,并分析了各自适用的范围,最后简单介绍了目前Key-Value数据库的企业级应用情况和几款开源的Key-Value数据库,并总结了使用Key-Value数据库之前要考虑的几点技术问题。

一种基于裸闪存的Key-Value数据库优化方法

近年来,非关系型的key-value数据库得到越来越广泛的应用.然而,目前主流的key-value数据库或者是基于磁盘设计的,或者是传统的基于文件系统和闪存转换层FTL来构建的,难以发挥闪存存储设备的特性,限制了I/O的并发性能,且垃圾回收过程复杂.设计并实现了一种基于裸闪存的keyvalue数据管理架构Flashkv,通过用户态下的管理单元进行空间管理和垃圾回收,充分利用了闪存设备内部的并发特性,并简化了垃圾回收过程,去除了传统文件系统和FTL中的冗余功能,缩短了I/O路径.提出了基于闪存特点的I/O调度技术,优化了闪存的读写延迟,提高了吞吐率;提出了用户态缓存管理技术,降低了数据写入量和频繁系统调用所带来的开销.测试结果表明,Flashkv性能是levelDB的1.9~2.2倍,写入量减少60%~65%.

基于操作历史图的分布式Key-Value数据库一致性检测算法

分布式数据库系统的副本机制在提高系统可靠性及性能的同时,导致了多副本数据管理的一致性问题;数据一致性的实现需要一致性协议模型来进行预防,也需要一致性检测算法对非一致数据进行检测。首先,对读写操作记录之间的时序关系、安全一致性及并行一致性原则等概念进行定义;其次,根据操作记录集合中读写操作之间的并行与时序关系,提取出操作记录集合向操作记录图转化的规则,并在此基础上设计了操作记录向历史记录图的转化算法;然后,以历史记录图为输入,设计了违反一致性查找算法,查找并返回图中所有违反安全与并行一致性读操作的集合;最后,基于Cassandra进行实验并将读写一致性设置为ONE,通过YCSB产生并行读写压力测试,与同类算法的对比实验验证了所提算法在功能与效率两方面的优越性。

基于SSD的Key-Value系统优化设计与实现

随着互联网技术的迅猛发展,越来越多的非结构化数据涌入到人们的生活中,为这些数据建立高效的索引面临极大的挑战.键值数据库Key-Value以其结构简单和高扩展性而引起人们的广泛关注,已成为海量数据存储系统中的重要组成部分.由于Key-Value系统对吞吐量要求较高,而基于Flash的固态硬盘(solid state drive,SSD)能够提供很高的随机读性能,在SSD上构建Key-Value系统已成为海量数据存储领域的一大研究热点.鉴于Flash具有非定点更新、寿命有限等特性,基于SSD的KeyValue系统必须针对Flash的特性作专门优化.以一种称为SkimpyStash的基于SSD的Key-Value系统为基础,提出了一种新的Key-Value系统低延迟存储系统(low latency store,LLStore).LLStore使用内存文件映射技术来减少针对SSD的IO请求,除此之外,针对SkimpyStash中低效的压缩策略,提出一种改进方法,可以在少量增加内存开销的情况下极大地减少查询时间.通过与原系统的性能比较实验,LLStore在平均查询时间上可以获得至少12%的加速.

云计算系统中Key-Value数据管理研究

Key-Value数据库是应用于云环境下的典型云存储系统,基于key-value数据模型的研究对大数据管理或称云数据管理系统提出了新的需求与挑战,成为人们关注的热点。本文对key-value数据模型与数据读写方式作了简单介绍,引入了key-value索引机制,重点讨论了key-value查询操作,给出了关于多维点查询的通用算法。

一种基于key-value数据库的快速地名地址输入提示方法

现有地址输入提示方法涉及标准地址和POI的研究较少,地址字符串的索引大多采用Trie(字典)树索引,但Trie树建立时内存消耗巨大,在面临海量数据时,问题更加突出。针对以上问题,提出一种基于key-value数据库的快速地名地址输入提示方法。该方法基于Trie树结构进行改进,降低了地址索引的复杂度;基于keyvalue数据库构建Trie树,避免了内存消耗巨大的问题。实验结果表明,基于key-value数据库构建的Trie树索引较基于内存构建的Trie树索引在事务响应性能和内存消耗方面具有明显的优势和效率。

基于Key-Value的云数据管理应用接口标准化研究

分析了基于Key-Value的云数据管理应用接口的标准化问题和面临的挑战,对现有基于Key-Value的云数据管理服务进行了调研,总结出包括数据模型和REST API等方面的异同,并提出未来的标准化工作建议。

分布式key-value系统错误污染检测

随着key-value存储系统的广泛使用,越来越多的研究开始关注分布式系统中的可信问题,其中一个重要的问题是,如何在系统被入侵或者管理员配置错误并运行一段时间后,检测出受污染的数据,从而可以在恢复错误数据的同时保留系统的合法更新.文中提出了一种基于key-value存储系统的错误污染检测方法,该方法允许在客户端不可信的前提下,检测客户端之间的污染扩散.文中设计了一种基于各服务器逻辑时钟的向量时钟,该时钟以用户关联操作为更新规则,记录了跨服务器的读写请求逻辑关系,用以进行错误污染跟踪;同时为了减少大规模系统中污染检测的开销,基于该向量时钟,文中进一步提出了一种在分布式系统中由操作序列构成有向无环图的污染分析方法.基于广泛使用的Voldmort key-value系统,文中实现了一个错误污染检测系统,Tracker Store.在集群测试环境下,文中对引入新的检测系统后产生的额外延迟开销进行了测试.

Key-Value型NoSQL本地存储系统研究

NoSQL系统因其高性能、高可扩展性的优势在大数据管理中得到广泛应用,而key-value(KV)模型则是NoSQL系统中使用最广泛的一种存储模型.KV型本地存储系统对于以机械磁盘为持久化存储的情形,存在许多性能优化技术,但这些优化技术面对当前的硬件发展新趋势,如多核处理器、大内存和低延迟闪存、非易失性内存NVM(Non-Volatile Memory)等,难以充分发挥新硬件的优势,如数据索引、并发控制、事务日志管理等技术在多核架构下存在多核扩展性问题,又如数据存储策略不适应闪存SSD(Solid State Drive)的新存储特性而产生了IO利用率低效的问题.针对多核处理器、大内存和闪存、NVM等硬件发展新趋势,文中面向当前的大数据应用背景,综述了KV型本地存储系统在索引技术、并发控制、事务日志管理和数据放置等核心模块上的最新优化技术和系统研究成果.从处理器、内存和持久化存储的角度概括了KV型本地存储系统当前存在的最优技术,总结了当前研究尚未解决的技术挑战,并对KV型本地存储系统在CPU缓存高效性、事务日志扩展性和高可用性等方面的研究进行了展望.

Utility-Improved Key-Value Data Collection with Local Differential Privacy for Mobile Devices

The structure of key-value data is a typical data structure generated by mobile devices.The collection and analysis of the data from mobile devices are critical for service providers to improve service quality.Nevertheless,collecting raw data,which may contain various per⁃sonal information,would lead to serious personal privacy leaks.Local differential privacy(LDP)has been proposed to protect privacy on the device side so that the server cannot obtain the raw data.However,existing mechanisms assume that all keys are equally sensitive,which can⁃not produce high-precision statistical results.A utility-improved data collection framework with LDP for key-value formed mobile data is pro⁃posed to solve this issue.More specifically,we divide the key-value data into sensitive and non-sensitive parts and only provide an LDPequivalent privacy guarantee for sensitive keys and all values.We instantiate our framework by using a utility-improved key value-unary en⁃coding(UKV-UE)mechanism based on unary encoding,with which our framework can work effectively for a large key domain.We then vali⁃date our mechanism which provides better utility and is suitable for mobile devices by evaluating it in two real datasets.Finally,some pos⁃sible future research directions are envisioned.

A Surfing Concurrence Transaction Model for Key-Value NoSQL Databases

As more and more application systems related to big data were developed, NoSQL (Not Only SQL) database systems are becoming more and more popular. In order to add transaction features for some NoSQL database systems, many scholars have tried different techniques. Unfortunately, there is a lack of research on Redis’s transaction in the existing literatures. This paper proposes a transaction model for key-value NoSQL databases including Redis to make possible allowing users to access data in the ACID (Atomicity, Consistency, Isolation and Durability) way, and this model is vividly called the surfing concurrence transaction model. The architecture, important features and implementation principle are described in detail. The key algorithms also were given in the form of pseudo program code, and the performance also was evaluated. With the proposed model, the transactions of Key-Value NoSQL databases can be performed in a lock free and MVCC (Multi-Version Concurrency Control) free manner. This is the result of further research on the related topic, which fills the gap ignored by relevant scholars in this field to make a little contribution to the further development of NoSQL technology.

Key-Value系统LSM模型实现技术的研究

随着时代的进步,社交网络的普及,Web2.0时代的到来,互联网的数据量呈现出爆炸式的增长。传统的数据库模型在高并发性、可扩展性等方面表现不佳,对于一些大数据的应用,传统的数据库模型无法胜任。为了解决这方面的需求,近些年来NoSQL技术快速的发展。其中Key-Value存储系统是最常见和简单的NoSQL系统,本文就将对Key-Value类型的系统实现技术进行研究和讨论。

SQL or NoSQL—Practical Aspect and Rational behind Choosing Data Stores

Data storage solutions are a crucial aspect of any application, significantly impacting data management and system performance. This article explores the rationale behind utilizing both SQL and NoSQL databases, addressing key questions about when each type is preferable. The background emphasizes the importance of selecting the appropriate database technology to meet specific application requirements. The purpose of this research is to provide a comprehensive guide for choosing between SQL and NoSQL databases based on various factors, including workload characteristics, scalability needs, and consistency requirements. To achieve this, we examine different strategies for implementing SQL and NoSQL databases in large-scale distributed applications and systems. The research method involves a comparative analysis of the features, advantages, and limitations of both database types. We specifically focus on scenarios involving read-heavy versus write-heavy systems and the trade-offs between availability and consistency. The results of this research indicate that SQL databases, with their relational structure and ACID compliance, are ideal for applications requiring complex queries and data integrity. In contrast, NoSQL databases, offering schema flexibility and horizontal scalability, are better suited for managing extensive datasets and high-velocity data ingestion. In conclusion, the selection of a database depends on the specific needs of the application. SQL databases are preferred for transactional systems with complex relationships, while NoSQL databases excel in scenarios demanding flexibility and scalability. The study provides insights into hybrid approaches, leveraging both database types to optimize system performance.

Zhaci and Nuonuo Kids Bookstore

Nuonuo Kids Bookstore is a dreamlike reading kingdom,established by Zhaci and his team with boundless enthusiasm and considerate design.In such a magical space,laughters and exclamations of kids harmonize into the most beautiful melody.Here,they learn to think,feel,love and also be loved.

基于双边市场理论的苹果App Store模式研究

从双边市场理论的定义、识别条件、特征、分类和平台企业战略等角度,分析了苹果Appstore模式的特征及平台战略。

32位RISC微处理器Load/Store部件的设计

简要介绍了一个32位RISC微处理器芯片ARS03的体系结构,阐述了处理器内部各个模块的功能。着重讨论了其中的Load/Store部件的设计,以及对关键路径的优化措施。Verilog仿真、综合和静态时序分析的的结果表明设计达到了预定要求。目前ARS03微处理器已经进入后端流程,不久就将使用0.25微米的工艺进行流片。