OPeNDAP(Open-source Project for a Network Data Access Protocol)代表开放源代码的网络数据访问协议。OPeNDAP协议提供一种通过网络请求和提供数据的方法,让所有用户使用熟悉的程序直接存取需要的数据。文章介绍了OPeNDAP常用服务器...OPeNDAP(Open-source Project for a Network Data Access Protocol)代表开放源代码的网络数据访问协议。OPeNDAP协议提供一种通过网络请求和提供数据的方法,让所有用户使用熟悉的程序直接存取需要的数据。文章介绍了OPeNDAP常用服务器,描述了基于OPeNDAP的南海物理海洋科学数据共享平台的设计与搭建:采用GDS(GrADS Data Server)、Dapper、THREDDS(Thematic Realtime Environmental Distributed Data Services)等服务器来实现海风数据、海流数据、海浪数据、Argo数据和潮位数据等南海物理海洋数据的OPeNDAP服务,最后介绍目前提供的OPeNDAP服务形式。希望借助OPeNDAP来实现分布、异构环境下南海物理海洋数据的网络无缝共享之目标。展开更多
Based on MATRIXx, a universal real-time visual distributed simulation system is developed. The system can receive different input data from network or local terminal. Application models in the simulation modules can a...Based on MATRIXx, a universal real-time visual distributed simulation system is developed. The system can receive different input data from network or local terminal. Application models in the simulation modules can automatically get such data to be analyzed and calculated, and then produce real-time simulation control information. Meanwhile, this paper designs relevant simulation components to implement the input and output data, which can guarantee the real-time and universal of the data transmission. Result of the experimental system shows that the real-time performance of the simulation is perfect.展开更多
To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the sch...To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.展开更多
This paper formally defines and analyses the new notion of correctness called quasi serializability, and then outlines corresponding concurrency control protocol QDHP for distributed real-time databases. Finally, thro...This paper formally defines and analyses the new notion of correctness called quasi serializability, and then outlines corresponding concurrency control protocol QDHP for distributed real-time databases. Finally, through a series of simulation studies, it shows that using the new concurrency control protocol the performance of distributed real-time databases can be much improved.展开更多
Mobile agents are able to migrate among machines to achieve their tasks. This feature is attractive to design, implement, and maintain distributed systems because we can implement both client-side and server-side prog...Mobile agents are able to migrate among machines to achieve their tasks. This feature is attractive to design, implement, and maintain distributed systems because we can implement both client-side and server-side programming in one mobile agent. However, it involves the increase of data traffic for mobile agent migrations. In this paper, we propose program code caching to reduce the data traffic caused by mobile agent migrations. A mobile agent consists of many program codes that define a task executed in each machine they migrate; thus, the mobile agent migration involves the transfer of their program codes. Therefore, our method reduces the number of the transfer of program codes by using program code cache. We have implemented our method on a mobile agent framework called Maglog and conducted experiments on a meeting scheduling system.展开更多
According to the different equipment, different system and heterogeneous database have be information ＂isolated island＂ problem, and the data of equipments can be updated in real time on the business node. The paper...According to the different equipment, different system and heterogeneous database have be information ＂isolated island＂ problem, and the data of equipments can be updated in real time on the business node. The paper proposes a program of data synchronization platform based on J2EE （JMS） and XML, and detailed analysis and description of the workflow system, its frame structure and the key technology. Practice shows that this scheme has the advantages of convenient and real-time etc..展开更多
Since the early 1990, significant progress in database technology has provided new platform for emerging new dimensions of data engineering. New models were introduced to utilize the data sets stored in the new genera...Since the early 1990, significant progress in database technology has provided new platform for emerging new dimensions of data engineering. New models were introduced to utilize the data sets stored in the new generations of databases. These models have a deep impact on evolving decision-support systems. But they suffer a variety of practical problems while accessing real-world data sources. Specifically a type of data storage model based on data distribution theory has been increasingly used in recent years by large-scale enterprises, while it is not compatible with existing decision-support models. This data storage model stores the data in different geographical sites where they are more regularly accessed. This leads to considerably less inter-site data transfer that can reduce data security issues in some circumstances and also significantly improve data manipulation transactions speed. The aim of this paper is to propose a new approach for supporting proactive decision-making that utilizes a workable data source management methodology. The new model can effectively organize and use complex data sources, even when they are distributed in different sites in a fragmented form. At the same time, the new model provides a very high level of intellectual management decision-support by intelligent use of the data collections through utilizing new smart methods in synthesizing useful knowledge. The results of an empirical study to evaluate the model are provided.展开更多
The rapid development of Internet of Things imposes new requirements on the data mining system, due to the weak capability of traditional distributed networking data mining. To meet the needs of the Internet of Things...The rapid development of Internet of Things imposes new requirements on the data mining system, due to the weak capability of traditional distributed networking data mining. To meet the needs of the Internet of Things, this paper proposes a novel distributed data-mining model to realize the seamless access between cloud computing and distributed data mining. The model is based on the cloud computing architecture, which belongs to the type of incredible nodes.展开更多
The temperature is an important factor affecting the stability of electron beam orbits.For almost all accelerators,a well-working cooling system and the monitoring system are needed to keep the temperature constant. I...The temperature is an important factor affecting the stability of electron beam orbits.For almost all accelerators,a well-working cooling system and the monitoring system are needed to keep the temperature constant. In this paper,the temperature monitor system for HLS based on EPICS,including its hardware,software and history database,is briefly introduced.It has been indicated that the system satisfies the requirement of machine study and operation.展开更多
With supercomputers developing towards exascale, the number of compute cores increases dramatically, making more complex and larger-scale applications possible. The input/output （I/O） requirements of large-scale app...With supercomputers developing towards exascale, the number of compute cores increases dramatically, making more complex and larger-scale applications possible. The input/output （I/O） requirements of large-scale applications, workflow applications, and their checkpointing include substantial bandwidth and an extremely low latency, posing a serious challenge to high performance computing （HPC） storage systems. Current hard disk drive （HDD） based underlying storage systems are becoming more and more incompetent to meet the requirements of next-generation exascale supercomputers. To rise to the challenge, we propose a hierarchical hybrid storage system, on-line and near-line file system （ONFS）. It leverages dynamic random access memory （DRAM） and solid state drive （SSD） in compute nodes, and HDD in storage servers to build a three-level storage system in a unified namespace. It supports portable operating system interface （POSIX） semantics, and provides high bandwidth, low latency, and huge storage capacity. In this paper, we present the technical details on distributed metadata management, the strategy of memory borrow and return, data consistency, parallel access control, and mechanisms guiding downward and upward migration in ONFS. We implement an ONFS prototype on the TH-1A supercomputer, and conduct experiments to test its I/O performance and scalability. The results show that the bandwidths of single-thread and multi-thread ＇read＇/＇write＇ are 6-fold and 5-fold better than HDD-based Lustre, respectively. The I/O bandwidth of data-intensive applications in ONFS can be 6.35 timcs that in Lustre.展开更多
文摘OPeNDAP(Open-source Project for a Network Data Access Protocol)代表开放源代码的网络数据访问协议。OPeNDAP协议提供一种通过网络请求和提供数据的方法,让所有用户使用熟悉的程序直接存取需要的数据。文章介绍了OPeNDAP常用服务器,描述了基于OPeNDAP的南海物理海洋科学数据共享平台的设计与搭建:采用GDS(GrADS Data Server)、Dapper、THREDDS(Thematic Realtime Environmental Distributed Data Services)等服务器来实现海风数据、海流数据、海浪数据、Argo数据和潮位数据等南海物理海洋数据的OPeNDAP服务,最后介绍目前提供的OPeNDAP服务形式。希望借助OPeNDAP来实现分布、异构环境下南海物理海洋数据的网络无缝共享之目标。
文摘Based on MATRIXx, a universal real-time visual distributed simulation system is developed. The system can receive different input data from network or local terminal. Application models in the simulation modules can automatically get such data to be analyzed and calculated, and then produce real-time simulation control information. Meanwhile, this paper designs relevant simulation components to implement the input and output data, which can guarantee the real-time and universal of the data transmission. Result of the experimental system shows that the real-time performance of the simulation is perfect.
基金supported by the National Grand Fundamental Research of China (973 Program) under Grant No. 2011CB302601the National High Technology Research and Development of China (863 Program) under GrantNo. 2013AA01A213+2 种基金the National Natural Science Foundation of China under Grant No. 60873215the Natural Science Foundation for Distinguished Young Scholars of Hunan Province under Grant No. S2010J5050Specialized Research Fund for the Doctoral Program of Higher Education under Grant No. 20124307110015
文摘To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.
基金the National Natural Science Foundation of China and the Commission of Science,Technokgy and Industry for National Defense
文摘This paper formally defines and analyses the new notion of correctness called quasi serializability, and then outlines corresponding concurrency control protocol QDHP for distributed real-time databases. Finally, through a series of simulation studies, it shows that using the new concurrency control protocol the performance of distributed real-time databases can be much improved.
文摘Mobile agents are able to migrate among machines to achieve their tasks. This feature is attractive to design, implement, and maintain distributed systems because we can implement both client-side and server-side programming in one mobile agent. However, it involves the increase of data traffic for mobile agent migrations. In this paper, we propose program code caching to reduce the data traffic caused by mobile agent migrations. A mobile agent consists of many program codes that define a task executed in each machine they migrate; thus, the mobile agent migration involves the transfer of their program codes. Therefore, our method reduces the number of the transfer of program codes by using program code cache. We have implemented our method on a mobile agent framework called Maglog and conducted experiments on a meeting scheduling system.
文摘According to the different equipment, different system and heterogeneous database have be information ＂isolated island＂ problem, and the data of equipments can be updated in real time on the business node. The paper proposes a program of data synchronization platform based on J2EE （JMS） and XML, and detailed analysis and description of the workflow system, its frame structure and the key technology. Practice shows that this scheme has the advantages of convenient and real-time etc..
文摘Since the early 1990, significant progress in database technology has provided new platform for emerging new dimensions of data engineering. New models were introduced to utilize the data sets stored in the new generations of databases. These models have a deep impact on evolving decision-support systems. But they suffer a variety of practical problems while accessing real-world data sources. Specifically a type of data storage model based on data distribution theory has been increasingly used in recent years by large-scale enterprises, while it is not compatible with existing decision-support models. This data storage model stores the data in different geographical sites where they are more regularly accessed. This leads to considerably less inter-site data transfer that can reduce data security issues in some circumstances and also significantly improve data manipulation transactions speed. The aim of this paper is to propose a new approach for supporting proactive decision-making that utilizes a workable data source management methodology. The new model can effectively organize and use complex data sources, even when they are distributed in different sites in a fragmented form. At the same time, the new model provides a very high level of intellectual management decision-support by intelligent use of the data collections through utilizing new smart methods in synthesizing useful knowledge. The results of an empirical study to evaluate the model are provided.
文摘The rapid development of Internet of Things imposes new requirements on the data mining system, due to the weak capability of traditional distributed networking data mining. To meet the needs of the Internet of Things, this paper proposes a novel distributed data-mining model to realize the seamless access between cloud computing and distributed data mining. The model is based on the cloud computing architecture, which belongs to the type of incredible nodes.
文摘The temperature is an important factor affecting the stability of electron beam orbits.For almost all accelerators,a well-working cooling system and the monitoring system are needed to keep the temperature constant. In this paper,the temperature monitor system for HLS based on EPICS,including its hardware,software and history database,is briefly introduced.It has been indicated that the system satisfies the requirement of machine study and operation.
基金Project supported by the National Key Research and Development Program of China(No.2016YFB0200402)
文摘With supercomputers developing towards exascale, the number of compute cores increases dramatically, making more complex and larger-scale applications possible. The input/output （I/O） requirements of large-scale applications, workflow applications, and their checkpointing include substantial bandwidth and an extremely low latency, posing a serious challenge to high performance computing （HPC） storage systems. Current hard disk drive （HDD） based underlying storage systems are becoming more and more incompetent to meet the requirements of next-generation exascale supercomputers. To rise to the challenge, we propose a hierarchical hybrid storage system, on-line and near-line file system （ONFS）. It leverages dynamic random access memory （DRAM） and solid state drive （SSD） in compute nodes, and HDD in storage servers to build a three-level storage system in a unified namespace. It supports portable operating system interface （POSIX） semantics, and provides high bandwidth, low latency, and huge storage capacity. In this paper, we present the technical details on distributed metadata management, the strategy of memory borrow and return, data consistency, parallel access control, and mechanisms guiding downward and upward migration in ONFS. We implement an ONFS prototype on the TH-1A supercomputer, and conduct experiments to test its I/O performance and scalability. The results show that the bandwidths of single-thread and multi-thread ＇read＇/＇write＇ are 6-fold and 5-fold better than HDD-based Lustre, respectively. The I/O bandwidth of data-intensive applications in ONFS can be 6.35 timcs that in Lustre.
