Data Utilization-Based Adaptive Data Management Method for Distributed Storage System in WAN Environment

Recently,research on a distributed storage system that efficiently manages a large amount of data has been actively conducted following data production and demand increase.Physical expansion limits exist for traditional standalone storage systems,such as I/O and file system capacity.However,the existing distributed storage system does not consider where data is consumed and is more focused on data dissemination and optimizing the lookup cost of data location.And this leads to system performance degradation due to low locality occurring in a Wide Area Network(WAN)environment with high network latency.This problem hinders deploying distributed storage systems to multiple data centers over WAN.It lowers the scalability of distributed storage systems to accommodate data storage needs.This paper proposes a method for distributing data in a WAN environment considering network latency and data locality to solve this problem and increase overall system performance.The proposed distributed storage method monitors data utilization and locality to classify data temperature as hot,warm,and cold.With assigned data temperature,the proposed algorithm adaptively selects the appropriate data center and places data accordingly to overcome the excess latency from the WAN environment,leading to overall system performance degradation.This paper also conducts simulations to evaluate the proposed and existing distributed storage methods.The result shows that our proposed method reduced latency by 38%compared to the existing method.Therefore,the proposed method in this paper can be used in large-scale distributed storage systems over a WAN environment to improve latency and performance compared to existing methods,such as consistent hashing.

DESIGN OF EXACT REGENERATING HIERARCHICAL CODE FOR DISTRIBUTED STORAGE SYSTEM

Erasure code is widely used as the redundancy scheme in distributed storage system. When a storage node fails, the repair process often requires to transfer a large amount of data. Regenerating code and hierarchical code are two classes of codes proposed to reduce the repair bandwidth cost. Regenerating codes reduce the amount of data transferred by each helping node, while hierarchical codes reduce the number of nodes participating in the repair process. In this paper, we propose a "sub-code nesting framework" to combine them together. The resulting regenerating hierarchical code has low repair degree as hierarchical code and lower repair cost than hierarchical code. Our code can achieve exact regeneration of the failed node, and has the additional property of low updating complexity.

The Reliability Design and Availability Analysis of a Distributed Storage System

Current distributed parallel file systems and database systems can not satisfy the demands of data-intensive applications, such as storage capacity, access performance, reliability, scalability, and so on. Cluster-based storage sys tems have some shortcomings, too. To solve this kind of problems, a novel PC storage cluster solution is proposed, a distributed storage system based on 3-tiered agent architecture is designed, the system reliability model based on the masterslave backup mode is built, and the system availability is analyzed with the Markov model. According to the system availability formula and the values of the system parameters, the novel system can provide higher reliability and availability to satisfy users＇ requirements,

Grid Side Distributed Energy Storage Cloud Group End Region Hierarchical Time-Sharing Configuration Algorithm Based onMulti-Scale and Multi Feature Convolution Neural Network

There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.

Distributed Secure Storage Scheme Based on Sharding Blockchain

Distributed storage can store data in multiple devices or servers to improve data security.However,in today’s explosive growth of network data,traditional distributed storage scheme is faced with some severe challenges such as insufficient performance,data tampering,and data lose.A distributed storage scheme based on blockchain has been proposed to improve security and efficiency of traditional distributed storage.Under this scheme,the following improvements have been made in this paper.This paper first analyzes the problems faced by distributed storage.Then proposed to build a new distributed storage blockchain scheme with sharding blockchain.The proposed scheme realizes the partitioning of the network and nodes by means of blockchain sharding technology,which can improve the efficiency of data verification between nodes.In addition,this paper uses polynomial commitment to construct a new verifiable secret share scheme called PolyVSS.This new scheme is one of the foundations for building our improved distributed storage blockchain scheme.Compared with the previous scheme,our new scheme does not require a trusted third party and has some new features such as homomorphic and batch opening.The security of VSS can be further improved.Experimental comparisons show that the proposed scheme significantly reduces storage and communication costs.

Security Analysis on a Remote Data Auditing Scheme for Distributed Storage Systems

Remote data auditing becomes critical to ensure the storage reliability in distributed cloud storage.Recently,Le et al proposed an efficient private data auditing scheme NC-Audit designed for regenerating codes,which claimed that NC-Audit can effectively realize privacy-preserving data auditing for distributed storage systems.However,our analysis shows that NC-Audit is not secure for that the adversarial cloud can forge some illegal blocks to cheat the auditor successfully with a high probability even without storing the user’s whole data,when the coding field is large enough.

SA-RSR:a read-optimal data recovery strategy for XOR-coded distributed storage systems

To ensure the reliability and availability of data,redundancy strategies are always required for distributed storage systems.Erasure coding,one of the representative redundancy strategies,has the advantage of low storage overhead,which facilitates its employment in distributed storage systems.Among the various erasure coding schemes,XOR-based erasure codes are becoming popular due to their high computing speed.When a single-node failure occurs in such coding schemes,a process called data recovery takes place to retrieve the failed node’s lost data from surviving nodes.However,data transmission during the data recovery process usually requires a considerable amount of time.Current research has focused mainly on reducing the amount of data needed for data recovery to reduce the time required for data transmission,but it has encountered problems such as significant complexity and local optima.In this paper,we propose a random search recovery algorithm,named SA-RSR,to speed up single-node failure recovery of XOR-based erasure codes.SA-RSR uses a simulated annealing technique to search for an optimal recovery solution that reads and transmits a minimum amount of data.In addition,this search process can be done in polynomial time.We evaluate SA-RSR with a variety of XOR-based erasure codes in simulations and in a real storage system,Ceph.Experimental results in Ceph show that SA-RSR reduces the amount of data required for recovery by up to 30.0%and improves the performance of data recovery by up to 20.36%compared to the conventional recovery method.

DCRL-KG: Distributed Multi-Modal Knowledge Graph Retrieval Platform Based on Collaborative Representation Learning

The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms.Image and text descriptions added to the knowledge graph enrich the node information,which accounts for the advantage of the multi-modal knowledge graph.In the field of cross-modal retrieval platforms,multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational infor-mation provided by knowledge graphs.The representation learning method is sig-nificant to the application of multi-modal knowledge graphs.This paper proposes a distributed collaborative vector retrieval platform(DCRL-KG)using the multi-modal knowledge graph VisualSem as the foundation to achieve efficient and high-precision multimodal data retrieval.Firstly,use distributed technology to classify and store the data in the knowledge graph to improve retrieval efficiency.Secondly,this paper uses BabelNet to expand the knowledge graph through multi-ple filtering processes and increase the diversification of information.Finally,this paper builds a variety of retrieval models to achieve the fusion of retrieval results through linear combination methods to achieve high-precision language retrieval and image retrieval.The paper uses sentence retrieval and image retrieval experi-ments to prove that the platform can optimize the storage structure of the multi-modal knowledge graph and have good performance in multi-modal space.

Locally Minimum Storage Regenerating Codes in Distributed Cloud Storage Systems

In distributed cloud storage systems, inevitably there exist multiple node failures at the same time. The existing methods of regenerating codes, including minimum storage regenerating(MSR) codes and minimum bandwidth regenerating(MBR) codes, are mainly to repair one single or several failed nodes, unable to meet the repair need of distributed cloud storage systems. In this paper, we present locally minimum storage regenerating(LMSR) codes to recover multiple failed nodes at the same time. Specifically, the nodes in distributed cloud storage systems are divided into multiple local groups, and in each local group(4, 2) or(5, 3) MSR codes are constructed. Moreover, the grouping method of storage nodes and the repairing process of failed nodes in local groups are studied. Theoretical analysis shows that LMSR codes can achieve the same storage overhead as MSR codes. Furthermore, we verify by means of simulation that, compared with MSR codes, LMSR codes can reduce the repair bandwidth and disk I/O overhead effectively.

Distributed energy storage node controller and control strategy based on energy storage cloud platform architecture

Based on the energy storage cloud platform architecture,this study considers the extensive configuration of energy storage devices and the future large-scale application of electric vehicles at the customer side to build a new mode of smart power consumption with a flexible interaction,smooth the peak/valley difference of the load side power,and improve energy efficiency.A plug and play device for customer-side energy storage and an internet-based energy storage cloud platform are developed herein to build a new intelligent power consumption mode with a flexible interaction suitable for ordinary customers.Based on the load perception of the power grid,this study aims to investigate the operating state and service life of distributed energy storage devices.By selecting an integrated optimal control scheme,this study designs a kind of energy optimization and deployment strategy for stratified partition to reduce the operating cost of the energy storage device on the client side.The effectiveness of the system and the control strategy is verified through the Suzhou client-side distributed energy storage demonstration project.

On Distributed Object Storage Architecture Based on Mimic Defense

With the advent of the era of big data,cloud computing,Internet of things,and other information industries continue to develop.There is an increasing amount of unstructured data such as pictures,audio,and video on the Internet.And the distributed object storage system has become the mainstream cloud storage solution.With the increasing number of distributed applications,data security in the distributed object storage system has become the focus.For the distributed object storage system,traditional defenses are means that fix discovered system vulnerabilities and backdoors by patching,or means to modify the corresponding structure and upgrade.However,these two kinds of means are hysteretic and hardly deal with unknown security threats.Based on mimic defense theory,this paper constructs the principle framework of the distributed object storage system and introduces the dynamic redundancy and heterogeneous function in the distributed object storage system architecture,which increases the attack cost,and greatly improves the security and availability of data.

Robust and Trustworthy Data Sharing Framework Leveraging On-Chain and Off-Chain Collaboration

The proliferation of Internet of Things(IoT)systems has resulted in the generation of substantial data,presenting new challenges in reliable storage and trustworthy sharing.Conventional distributed storage systems are hindered by centralized management and lack traceability,while blockchain systems are limited by low capacity and high latency.To address these challenges,the present study investigates the reliable storage and trustworthy sharing of IoT data,and presents a novel system architecture that integrates on-chain and off-chain data manage systems.This architecture,integrating blockchain and distributed storage technologies,provides high-capacity,high-performance,traceable,and verifiable data storage and access.The on-chain system,built on Hyperledger Fabric,manages metadata,verification data,and permission information of the raw data.The off-chain system,implemented using IPFS Cluster,ensures the reliable storage and efficient access to massive files.A collaborative storage server is designed to integrate on-chain and off-chain operation interfaces,facilitating comprehensive data operations.We provide a unified access interface for user-friendly system interaction.Extensive testing validates the system’s reliability and stable performance.The proposed approach significantly enhances storage capacity compared to standalone blockchain systems.Rigorous reliability tests consistently yield positive outcomes.With average upload and download throughputs of roughly 20 and 30 MB/s,respectively,the system’s throughput surpasses the blockchain system by a factor of 4 to 18.

Low-Carbon Dispatching for Virtual Power Plant with Aggregated Distributed Energy Storage Considering Spatiotemporal Distribution of Cleanness Value

The scale of distributed energy resources is increasing,but imperfect business models and value transmission mechanisms lead to low utilization ratio and poor responsiveness.To address this issue,the concept of cleanness value of distributed energy storage(DES)is proposed,and the spatiotemporal distribution mechanism is discussed from the perspectives of electrical energy and cleanness.Based on this,an evaluation system for the environmental benefits of DES is constructed to balance the interests between the aggregator and the power system operator.Then,an optimal low-carbon dispatching for a virtual power plant(VPP)with aggregated DES is constructed,where-in energy value and cleanness value are both considered.To achieve the goal,a green attribute labeling method is used to establish a correlation constraint between the nodal carbon potential of the distribution network(DN)and DES behavior,but as a cost,it brings multiple nonlinear relationships.Subsequently,a solution method based on the convex envelope(CE)linear re-construction method is proposed for the multivariate nonlinear programming problem,thereby improving solution efficiency and feasibility.Finally,the simulation verification based on the IEEE 33-bus DN is conducted.The simulation results show that the multidimensional value recognition of DES motivates the willingness of resource users to respond.Meanwhile,resolving the impact of DES on the nodal carbon potential can effectively alleviate overcompensation of the cleanness value.

A Secured Message Transmission Protocol for Vehicular Ad Hoc Networks

Vehicular Ad hoc Networks(VANETs)become a very crucial addition in the Intelligent Transportation System(ITS).It is challenging for a VANET system to provide security services and parallelly maintain high throughput by utilizing limited resources.To overcome these challenges,we propose a blockchain-based Secured Cluster-based MAC(SCB-MAC)protocol.The nearby vehicles heading towards the same direction will form a cluster and each of the clusters has its blockchain to store and distribute the safety messages.The message which contains emergency information and requires Strict Delay Requirement(SDR)for transmission are called safety messages(SM).Cluster Members(CMs)sign SMs with their private keys while sending them to the blockchain to confirm authentication,integrity,and confidentiality of the message.A Certificate Authority(CA)is responsible for physical verification,key generation,and privacy preservation of the vehicles.We implemented a test scenario as proof of concept and tested the safety message transmission(SMT)protocol in a real-world platform.Computational and storage overhead analysis shows that the proposed protocol for SMT implements security,authentication,integrity,robustness,non-repudiation,etc.while maintaining the SDR.Messages that are less important compared to the SMs are called non-safety messages(NSM)and vehicles use RTS/CTS mechanism for NSM transmission.Numerical studies show that the proposed NSM transmission method maintains 6 times more throughput,2 times less delay and 125%less Packet Dropping Rate(PDR)than traditional MAC protocols.These results prove that the proposed protocol outperforms the traditional MAC protocols.

Massive Files Prefetching Model Based on LSTM Neural Network with Cache Transaction Strategy

In distributed storage systems,file access efficiency has an important impact on the real-time nature of information forensics.As a popular approach to improve file accessing efficiency,prefetching model can fetches data before it is needed according to the file access pattern,which can reduce the I/O waiting time and increase the system concurrency.However,prefetching model needs to mine the degree of association between files to ensure the accuracy of prefetching.In the massive small file situation,the sheer volume of files poses a challenge to the efficiency and accuracy of relevance mining.In this paper,we propose a massive files prefetching model based on LSTM neural network with cache transaction strategy to improve file access efficiency.Firstly,we propose a file clustering algorithm based on temporal locality and spatial locality to reduce the computational complexity.Secondly,we propose a definition of cache transaction according to files occurrence in cache instead of time-offset distance based methods to extract file block feature accurately.Lastly,we innovatively propose a file access prediction algorithm based on LSTM neural network which predict the file that have high possibility to be accessed.Experiments show that compared with the traditional LRU and the plain grouping methods,the proposed model notably increase the cache hit rate and effectively reduces the I/O wait time.

Coordinated control for voltage regulation of distribution network voltage regulation by distributed energy storage systems

With more and more distributed photovoltaic(PV)plants access to the distribution system,whose structure is changing and becoming an active network.The traditional methods of voltage regulation may hardly adapt to this new situation.To address this problem,this paper presents a coordinated control method of distributed energy storage systems(DESSs)for voltage regulation in a distribution network.The influence of the voltage caused by the PV plant is analyzed in a simple distribution feeder at first.The voltage regulation areas corresponding to DESSs are divided by calculating and comparing the voltage sensitivity matrix.Then,a coordinated voltage control strategy is proposed for the DESSs.Finally,the simulation results of the IEEE 33-bus radial distribution network verify the effectiveness of the proposed coordinated control method.

Progress in Theoretical Research on Reliability for China's Space Engineering

With the support of the Fundamental Reliability Theoretical Research (FRTR) Foundation of the Quality Control Bureau of Ministry of Astronautics (MOA), PRC, 9 Chinese institutes and universities have worked for years on reliability statistics problems pending to be solved in space research and development. This paper gives a brief review of our main research results, including (1) Results on Normal Distributions; (2) Results on Weibull Distributions; (3) Results on the Synthesisof System Reliability-Theoretical Method; (4) Results on the Synthesis of System Reliability-Approximation Method: Binomial Distribution, Exponential Distribution, Weibull Distribution, Parallel System, General Cases; (5) Structual Reliability; (6) Zero-Failure Reliability Estimation; (7) Storage Life and Others. All these results can be acquired from the Quality Control Bureau of the Ministry of Aero-Space Industry (MAS).

Identity-based searchable attribute signcryption in lattice for a blockchain-based medical system

Electronic healthcare systems can offer convenience but face the risk of data forgery and information leakage.To solve these issues,we propose an identity-based searchable attribute signcryption in lattice for a blockchain-based medical system(BCMS-LIDSASC).BCMS-LIDSASC achieves decentralization and anti-quantum security in the blockchain environment,and provides fine-grained access control and searchability.Furthermore,smart contracts are used to replace traditional trusted third parties,and the interplanetary file system(IPFS)is used for ciphertext storage to alleviate storage pressure on the blockchain.Compared to other schemes,BCMS-LIDSASC requires smaller key size and less storage,and has lower computation cost.It contributes to secure and efficient management of medical data and can protect patient privacy and ensure the integrity of electronic healthcare systems.

Constrained coordinated distributed control of smart grid with asynchronous information exchange

Smart grid constrained optimal control is a complex issue due to the constant growth of grid complexity and the large volume of data available as input to smart device control.In this context,traditional centralized control paradigms may suffer in terms of the timeliness of optimization results due to the volume of data to be processed and the delayed asynchronous nature of the data transmission.To address these limits of centralized control,this paper presents a coordinated,distributed algorithm based on distributed,local controllers and a central coordinator for exchanging summarized global state information.The proposed model for exchanging global state information is resistant to fluctuations caused by the inherent interdependence between local controllers,and is robust to delays in information exchange.In addition,the algorithm features iterative refinement of local state estimations that is able to improve local controller ability to operate within network constraints.Application of the proposed coordinated,distributed algorithm through simulation shows its effectiveness in optimizing a global goal within a complex distribution system operating under constraints,while ensuring network operation stability under varying levels of information exchange delay,and with a range of network sizes.

A Ceph-based storage strategy for big gridded remote sensing data

When using distributed storage systems to store gridded remote sensing data in large,distributed clusters,most solutions utilize big table index storage strategies.However,in practice,the performance of big table index storage strategies degrades as scenarios become more complex,and the reasons for this phenomenon are analyzed in this paper.To improve the read and write performance of distributed gridded data storage,this paper proposes a storage strategy based on Ceph software.The strategy encapsulates remote sensing images in the form of objects through a metadata management strategy to achieve the spatiotemporal retrieval of gridded data,finding the cluster location of gridded data through hash-like calculations.The method can effectively achieve spatial operation support in the clustered database and at the same time enable fast random read and write of the gridded data.Random write and spatial query experiments proved the feasibility,effectiveness,and stability of this strategy.The experiments prove that the method has higher stability than,and that the average query time is 38%lower than that for,the large table index storage strategy,which greatly improves the storage and query efficiency of gridded images.