Based on the concrete conditions of earthquake data in the west of China, East China and SOuth China, we studied the completeness of data in these regions by suitable methods to local conditions. Otherwise, we roughly...Based on the concrete conditions of earthquake data in the west of China, East China and SOuth China, we studied the completeness of data in these regions by suitable methods to local conditions. Otherwise, we roughly estimated monitoring capability of local networks in China since 1970 and some outlying regions where the data is lack. Finally, we gave the regional distribution of the beginning years since which the data for different magnitude intervals are largely complete in the Chinese mainland.展开更多
The smart grid is an evolving critical infrastructure,which combines renewable energy and the most advanced information and communication technologies to provide more economic and secure power supply services.To cope ...The smart grid is an evolving critical infrastructure,which combines renewable energy and the most advanced information and communication technologies to provide more economic and secure power supply services.To cope with the intermittency of ever-increasing renewable energy and ensure the security of the smart grid,state estimation,which serves as a basic tool for understanding the true states of a smart grid,should be performed with high frequency.More complete system state data are needed to support high-frequency state estimation.The data completeness problem for smart grid state estimation is therefore studied in this paper.The problem of improving data completeness by recovering highfrequency data from low-frequency data is formulated as a super resolution perception(SRP)problem in this paper.A novel machine-learning-based SRP approach is thereafter proposed.The proposed method,namely the Super Resolution Perception Net for State Estimation(SRPNSE),consists of three steps:feature extraction,information completion,and data reconstruction.Case studies have demonstrated the effectiveness and value of the proposed SRPNSE approach in recovering high-frequency data from low-frequency data for the state estimation.展开更多
Low quality of data is a serious problem in the new era of big data, which can severely reduce the usability of data, mislead or bias the querying, analyzing and mining, and leads to huge loss. Incomplete data is comm...Low quality of data is a serious problem in the new era of big data, which can severely reduce the usability of data, mislead or bias the querying, analyzing and mining, and leads to huge loss. Incomplete data is common in low quality data, and it is necessary to determine the data completeness of a dataset to provide hints for follow-up operations on it.Little existing work focuses on the completeness of a dataset, and such work views all missing values as unknown values. In this paper, we study how to determine real data completeness of a relational dataset. By taking advantage of given functional dependencies, we aim to determine some missing attribute values by other tuples and capture the really missing attribute cells. We propose a data completeness model, formalize the problem of determining the real data completeness of a relational dataset, and give a lower bound of the time complexity of this problem. Two optimal algorithms to determine the data completeness of a dataset for different cases are proposed. We empirically show the effectiveness and the scalability of our algorithms on both real-world data and synthetic data.展开更多
Nowadays,several research projects show interest in employing volunteered geographic information(VGI)to improve their systems through using up-to-date and detailed data.The European project CAP4Access is one of the su...Nowadays,several research projects show interest in employing volunteered geographic information(VGI)to improve their systems through using up-to-date and detailed data.The European project CAP4Access is one of the successful examples of such international-wide research projects that aims to improve the accessibility of people with restricted mobility using crowdsourced data.In this project,OpenStreetMap(OSM)is used to extend OpenRouteService,a well-known routing platform.However,a basic challenge that this project tackled was the incompleteness of OSM data with regards to certain information that is required for wheelchair accessibility(e.g.sidewalk information,kerb data,etc.).In this article,we present the results of initial assessment of sidewalk data in OSM at the beginning of the project as well as our approach in awareness raising and using tools for tagging accessibility data into OSM database for enriching the sidewalk data completeness.Several experiments have been carried out in different European cities,and discussion on the results of the experiments as well as the lessons learned are provided.The lessons learned provide recommendations that help in organizing better mapping party events in the future.We conclude by reporting on how and to what extent the OSM sidewalk data completeness in these study areas have benefited from the mapping parties by the end of the project.展开更多
With large-scale applications,the loss of power load data during transmission is inevitable.This paper proposes a data completion method considering the low rank property of the data.According to the low-rank property...With large-scale applications,the loss of power load data during transmission is inevitable.This paper proposes a data completion method considering the low rank property of the data.According to the low-rank property of data and numerical experiments,we find either the linear interpolation(LI)or the singular value decomposition(SVD)based method is superior to other methods depending on the smoothness of the data.We construct an index to measure the smoothness of data,and propose the SVDLI algorithm which adaptively selects different algorithms for data completion according to the index.Numerical simulations show that irrespective of the smoothness of data,the data complementing results of SVDLI are comparable to or better than the best of SVD or LI algorithms.The present study is verified using the measurements in China,and the public data of the Australian electricity distribution company and Lawrence Berkeley National Laboratory.展开更多
We present VCNet,a new deep learning approach for volume completion by synthesizing missing subvolumes.Our solution leverages a generative adversarial network(GAN)that learns to complete volumes using the adversarial ...We present VCNet,a new deep learning approach for volume completion by synthesizing missing subvolumes.Our solution leverages a generative adversarial network(GAN)that learns to complete volumes using the adversarial and volumetric losses.The core design of VCNet features a dilated residual block and long-term connection.During training,VCNet first randomly masks basic subvolumes(e.g.,cuboids,slices)from complete volumes and learns to recover them.Moreover,we design a two-stage algorithm for stabilizing and accelerating network optimization.Once trained,VCNet takes an incomplete volume as input and automatically identifies and fills in the missing subvolumes with high quality.We quantitatively and qualitatively test VCNet with volumetric data sets of various characteristics to demonstrate its effectiveness.We also compare VCNet against a diffusion-based solution and two GAN-based solutions.展开更多
文摘Based on the concrete conditions of earthquake data in the west of China, East China and SOuth China, we studied the completeness of data in these regions by suitable methods to local conditions. Otherwise, we roughly estimated monitoring capability of local networks in China since 1970 and some outlying regions where the data is lack. Finally, we gave the regional distribution of the beginning years since which the data for different magnitude intervals are largely complete in the Chinese mainland.
基金the Training Program of the Major Research Plan of the National Natural Science Foundation of China(91746118)the Shenzhen Municipal Science and Technology Innovation Committee Basic Research project(JCYJ20170410172224515)。
文摘The smart grid is an evolving critical infrastructure,which combines renewable energy and the most advanced information and communication technologies to provide more economic and secure power supply services.To cope with the intermittency of ever-increasing renewable energy and ensure the security of the smart grid,state estimation,which serves as a basic tool for understanding the true states of a smart grid,should be performed with high frequency.More complete system state data are needed to support high-frequency state estimation.The data completeness problem for smart grid state estimation is therefore studied in this paper.The problem of improving data completeness by recovering highfrequency data from low-frequency data is formulated as a super resolution perception(SRP)problem in this paper.A novel machine-learning-based SRP approach is thereafter proposed.The proposed method,namely the Super Resolution Perception Net for State Estimation(SRPNSE),consists of three steps:feature extraction,information completion,and data reconstruction.Case studies have demonstrated the effectiveness and value of the proposed SRPNSE approach in recovering high-frequency data from low-frequency data for the state estimation.
基金The work was supported by the National Basic Research 973 Program of China under Grant No. 2011CB036202 and the National Natural Science Foundation of China under Grant No. 61532015.
文摘Low quality of data is a serious problem in the new era of big data, which can severely reduce the usability of data, mislead or bias the querying, analyzing and mining, and leads to huge loss. Incomplete data is common in low quality data, and it is necessary to determine the data completeness of a dataset to provide hints for follow-up operations on it.Little existing work focuses on the completeness of a dataset, and such work views all missing values as unknown values. In this paper, we study how to determine real data completeness of a relational dataset. By taking advantage of given functional dependencies, we aim to determine some missing attribute values by other tuples and capture the really missing attribute cells. We propose a data completeness model, formalize the problem of determining the real data completeness of a relational dataset, and give a lower bound of the time complexity of this problem. Two optimal algorithms to determine the data completeness of a dataset for different cases are proposed. We empirically show the effectiveness and the scalability of our algorithms on both real-world data and synthetic data.
基金supported by the European Community’s Seventh Framework Programme[FP7/2007–2013],[Grant No 612096(CAP4Access)].
文摘Nowadays,several research projects show interest in employing volunteered geographic information(VGI)to improve their systems through using up-to-date and detailed data.The European project CAP4Access is one of the successful examples of such international-wide research projects that aims to improve the accessibility of people with restricted mobility using crowdsourced data.In this project,OpenStreetMap(OSM)is used to extend OpenRouteService,a well-known routing platform.However,a basic challenge that this project tackled was the incompleteness of OSM data with regards to certain information that is required for wheelchair accessibility(e.g.sidewalk information,kerb data,etc.).In this article,we present the results of initial assessment of sidewalk data in OSM at the beginning of the project as well as our approach in awareness raising and using tools for tagging accessibility data into OSM database for enriching the sidewalk data completeness.Several experiments have been carried out in different European cities,and discussion on the results of the experiments as well as the lessons learned are provided.The lessons learned provide recommendations that help in organizing better mapping party events in the future.We conclude by reporting on how and to what extent the OSM sidewalk data completeness in these study areas have benefited from the mapping parties by the end of the project.
文摘With large-scale applications,the loss of power load data during transmission is inevitable.This paper proposes a data completion method considering the low rank property of the data.According to the low-rank property of data and numerical experiments,we find either the linear interpolation(LI)or the singular value decomposition(SVD)based method is superior to other methods depending on the smoothness of the data.We construct an index to measure the smoothness of data,and propose the SVDLI algorithm which adaptively selects different algorithms for data completion according to the index.Numerical simulations show that irrespective of the smoothness of data,the data complementing results of SVDLI are comparable to or better than the best of SVD or LI algorithms.The present study is verified using the measurements in China,and the public data of the Australian electricity distribution company and Lawrence Berkeley National Laboratory.
基金This work was supported in part by the U.S.National Science Foundation through grants IIS-1455886,CNS-1629914,DUE-1833129,IIS-1955395,IIS-2101696,and OAC-2104158.The authors would like to thank the anonymous reviewers for their insightful comments.
文摘We present VCNet,a new deep learning approach for volume completion by synthesizing missing subvolumes.Our solution leverages a generative adversarial network(GAN)that learns to complete volumes using the adversarial and volumetric losses.The core design of VCNet features a dilated residual block and long-term connection.During training,VCNet first randomly masks basic subvolumes(e.g.,cuboids,slices)from complete volumes and learns to recover them.Moreover,we design a two-stage algorithm for stabilizing and accelerating network optimization.Once trained,VCNet takes an incomplete volume as input and automatically identifies and fills in the missing subvolumes with high quality.We quantitatively and qualitatively test VCNet with volumetric data sets of various characteristics to demonstrate its effectiveness.We also compare VCNet against a diffusion-based solution and two GAN-based solutions.