A novel data streams partitioning method is proposed to resolve problems of range-aggregation continuous queries over parallel streams for power industry.The first step of this method is to parallel sample the data,wh...A novel data streams partitioning method is proposed to resolve problems of range-aggregation continuous queries over parallel streams for power industry.The first step of this method is to parallel sample the data,which is implemented as an extended reservoir-sampling algorithm.A skip factor based on the change ratio of data-values is introduced to describe the distribution characteristics of data-values adaptively.The second step of this method is to partition the fluxes of data streams averagely,which is implemented with two alternative equal-depth histogram generating algorithms that fit the different cases：one for incremental maintenance based on heuristics and the other for periodical updates to generate an approximate partition vector.The experimental results on actual data prove that the method is efficient,practical and suitable for time-varying data streams processing.展开更多
Pushed by the Internet of Things(IoT)paradigm modern sensor networks monitor a wide range of phenomena,in areas such as environmental monitoring,health care,industrial processes,and smart cities.These networks provide...Pushed by the Internet of Things(IoT)paradigm modern sensor networks monitor a wide range of phenomena,in areas such as environmental monitoring,health care,industrial processes,and smart cities.These networks provide a continuous pulse of the almost infinite activities that are happening in the physical space and are thus,key enablers for a Digital Earth Nervous System.Nevertheless,the rapid processing of these sensor data streams still continues to challenge traditional data-handling solutions and new approaches are being requested.We propose a generic answer to this challenge,which has the potential to support any form of distributed real-time analysis.This neutral methodology follows a brokering approach to work with different kinds of data sources and uses web-based standards to achieve interoperability.As a proof of concept,we implemented the methodology to detect anomalies in real-time and applied it to the area of environmental monitoring.The developed system is capable of detecting anomalies,generating notifications,and displaying the recent situation to the user.展开更多
A partition checkpoint strategy based on data segment priority is presented to meet the timing constraints of the data and the transaction in embedded real-time main memory database systems（ERTMMDBS） as well as to r...A partition checkpoint strategy based on data segment priority is presented to meet the timing constraints of the data and the transaction in embedded real-time main memory database systems（ERTMMDBS） as well as to reduce the number of the transactions missing their deadlines and the recovery time.The partition checkpoint strategy takes into account the characteristics of the data and the transactions associated with it;moreover,it partitions the database according to the data segment priority and sets the corresponding checkpoint frequency to each partition for independent checkpoint operation.The simulation results show that the partition checkpoint strategy decreases the ratio of trans-actions missing their deadlines.展开更多
基金The High Technology Research Plan of Jiangsu Prov-ince (No.BG2004034)the Foundation of Graduate Creative Program ofJiangsu Province (No.xm04-36).
文摘A novel data streams partitioning method is proposed to resolve problems of range-aggregation continuous queries over parallel streams for power industry.The first step of this method is to parallel sample the data,which is implemented as an extended reservoir-sampling algorithm.A skip factor based on the change ratio of data-values is introduced to describe the distribution characteristics of data-values adaptively.The second step of this method is to partition the fluxes of data streams averagely,which is implemented with two alternative equal-depth histogram generating algorithms that fit the different cases：one for incremental maintenance based on heuristics and the other for periodical updates to generate an approximate partition vector.The experimental results on actual data prove that the method is efficient,practical and suitable for time-varying data streams processing.
基金European Commission and Generalitat Valenciana government[ACIF/2012/112]and[BEFPI/2014/067].
文摘Pushed by the Internet of Things(IoT)paradigm modern sensor networks monitor a wide range of phenomena,in areas such as environmental monitoring,health care,industrial processes,and smart cities.These networks provide a continuous pulse of the almost infinite activities that are happening in the physical space and are thus,key enablers for a Digital Earth Nervous System.Nevertheless,the rapid processing of these sensor data streams still continues to challenge traditional data-handling solutions and new approaches are being requested.We propose a generic answer to this challenge,which has the potential to support any form of distributed real-time analysis.This neutral methodology follows a brokering approach to work with different kinds of data sources and uses web-based standards to achieve interoperability.As a proof of concept,we implemented the methodology to detect anomalies in real-time and applied it to the area of environmental monitoring.The developed system is capable of detecting anomalies,generating notifications,and displaying the recent situation to the user.
基金Supported by the National Natural Science Foundation of China (60673128)
文摘A partition checkpoint strategy based on data segment priority is presented to meet the timing constraints of the data and the transaction in embedded real-time main memory database systems（ERTMMDBS） as well as to reduce the number of the transactions missing their deadlines and the recovery time.The partition checkpoint strategy takes into account the characteristics of the data and the transactions associated with it;moreover,it partitions the database according to the data segment priority and sets the corresponding checkpoint frequency to each partition for independent checkpoint operation.The simulation results show that the partition checkpoint strategy decreases the ratio of trans-actions missing their deadlines.
基金湖南省科技公关计划( the Key Technologies R&D Program of Hunan Province China under Grant No.05GK2002)+2 种基金湖南省自然科学基金( the Natural Science Foundation of Hunan Province of China under Grant No.03JJY6023)长沙市科技公关计划( the Key Technologies R&D Program of Changsha City China under Grant No.K06070001- 12)
