In several fields like financial dealing,industry,business,medicine,et cetera,Big Data(BD)has been utilized extensively,which is nothing but a collection of a huge amount of data.However,it is highly complicated alon...In several fields like financial dealing,industry,business,medicine,et cetera,Big Data(BD)has been utilized extensively,which is nothing but a collection of a huge amount of data.However,it is highly complicated along with time-consuming to process a massive amount of data.Thus,to design the Distribution Preserving Framework for BD,a novel methodology has been proposed utilizing Manhattan Distance(MD)-centered Partition Around Medoid(MD–PAM)along with Conjugate Gradient Artificial Neural Network(CG-ANN),which undergoes various steps to reduce the complications of BD.Firstly,the data are processed in the pre-processing phase by mitigating the data repetition utilizing the map-reduce function;subsequently,the missing data are handled by substituting or by ignoring the missed values.After that,the data are transmuted into a normalized form.Next,to enhance the classification performance,the data’s dimensionalities are minimized by employing Gaussian Kernel(GK)-Fisher Discriminant Analysis(GK-FDA).Afterwards,the processed data is submitted to the partitioning phase after transmuting it into a structured format.In the partition phase,by utilizing the MD-PAM,the data are partitioned along with grouped into a cluster.Lastly,by employing CG-ANN,the data are classified in the classification phase so that the needed data can be effortlessly retrieved by the user.To analogize the outcomes of the CG-ANN with the prevailing methodologies,the NSL-KDD openly accessible datasets are utilized.The experiential outcomes displayed that an efficient result along with a reduced computation cost was shown by the proposed CG-ANN.The proposed work outperforms well in terms of accuracy,sensitivity and specificity than the existing systems.展开更多
The conventional direct position determination(DPD) algorithm processes all received signals on a single sensor.When sensors have limited computational capabilities or energy storage,it is desirable to distribute th...The conventional direct position determination(DPD) algorithm processes all received signals on a single sensor.When sensors have limited computational capabilities or energy storage,it is desirable to distribute the computation among other sensors.A distributed adaptive DPD(DADPD)algorithm based on diffusion framework is proposed for emitter localization.Unlike the corresponding centralized adaptive DPD(CADPD) algorithm,all but one sensor in the proposed algorithm participate in processing the received signals and estimating the common emitter position,respectively.The computational load and energy consumption on a single sensor in the CADPD algorithm is distributed among other computing sensors in a balanced manner.Exactly the same iterative localization algorithm is carried out in each computing sensor,respectively,and the algorithm in each computing sensor exhibits quite similar convergence behavior.The difference of the localization and tracking performance between the proposed distributed algorithm and the corresponding CADPD algorithm is negligible through simulation evaluations.展开更多
Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data i...Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data increases at a pace that is faster than the increase in the big data processing capacity of clusters.Thus,distributed computing frameworks based on the MapReduce computing model are not adequate to support big data analysis tasks which often require running complex analytical algorithms on extremely big data sets in terabytes.In performing such tasks,these frameworks face three challenges:computational inefficiency due to high I/O and communication costs,non-scalability to big data due to memory limit,and limited analytical algorithms because many serial algorithms cannot be implemented in the MapReduce programming model.New distributed computing frameworks need to be developed to conquer these challenges.In this paper,we review MapReduce-type distributed computing frameworks that are currently used in handling big data and discuss their problems when conducting big data analysis.In addition,we present a non-MapReduce distributed computing framework that has the potential to overcome big data analysis challenges.展开更多
To enhance the cost-effectiveness of bulk hybrid AC-DC power systems and promote wind consumption,this paper proposes a two-stage risk-based robust reserve scheduling(RRRS)model.Different from traditional robust optim...To enhance the cost-effectiveness of bulk hybrid AC-DC power systems and promote wind consumption,this paper proposes a two-stage risk-based robust reserve scheduling(RRRS)model.Different from traditional robust optimization,the proposed model applies an adjustable uncertainty set rather than a fixed one.Thereby,the operational risk is optimized together with the dispatch schedules,with a reasonable admissible region of wind power obtained correspondingly.In addition,both the operational base point and adjustment capacity of tielines are optimized in the RRRS model,which enables reserve sharing among the connected areas to handle the significant wind uncertainties.Based on the alternating direction method of multipliers(ADMM),a fully distributed framework is presented to solve the RRRS model in a distributed way.A dynamic penalty factor adjustment strategy(DPA)is also developed and applied to enhance its convergence properties.Since only limited information needs to be exchanged during the solution process,the communication burden is reduced and regional information is protected.Case studies on the 2-area 12-bus system and 3-area 354-bus system illustrate the effectiveness of the proposed model and approach.展开更多
文摘In several fields like financial dealing,industry,business,medicine,et cetera,Big Data(BD)has been utilized extensively,which is nothing but a collection of a huge amount of data.However,it is highly complicated along with time-consuming to process a massive amount of data.Thus,to design the Distribution Preserving Framework for BD,a novel methodology has been proposed utilizing Manhattan Distance(MD)-centered Partition Around Medoid(MD–PAM)along with Conjugate Gradient Artificial Neural Network(CG-ANN),which undergoes various steps to reduce the complications of BD.Firstly,the data are processed in the pre-processing phase by mitigating the data repetition utilizing the map-reduce function;subsequently,the missing data are handled by substituting or by ignoring the missed values.After that,the data are transmuted into a normalized form.Next,to enhance the classification performance,the data’s dimensionalities are minimized by employing Gaussian Kernel(GK)-Fisher Discriminant Analysis(GK-FDA).Afterwards,the processed data is submitted to the partitioning phase after transmuting it into a structured format.In the partition phase,by utilizing the MD-PAM,the data are partitioned along with grouped into a cluster.Lastly,by employing CG-ANN,the data are classified in the classification phase so that the needed data can be effortlessly retrieved by the user.To analogize the outcomes of the CG-ANN with the prevailing methodologies,the NSL-KDD openly accessible datasets are utilized.The experiential outcomes displayed that an efficient result along with a reduced computation cost was shown by the proposed CG-ANN.The proposed work outperforms well in terms of accuracy,sensitivity and specificity than the existing systems.
基金supported by the National Natural Science Foundation of China(61101173)
文摘The conventional direct position determination(DPD) algorithm processes all received signals on a single sensor.When sensors have limited computational capabilities or energy storage,it is desirable to distribute the computation among other sensors.A distributed adaptive DPD(DADPD)algorithm based on diffusion framework is proposed for emitter localization.Unlike the corresponding centralized adaptive DPD(CADPD) algorithm,all but one sensor in the proposed algorithm participate in processing the received signals and estimating the common emitter position,respectively.The computational load and energy consumption on a single sensor in the CADPD algorithm is distributed among other computing sensors in a balanced manner.Exactly the same iterative localization algorithm is carried out in each computing sensor,respectively,and the algorithm in each computing sensor exhibits quite similar convergence behavior.The difference of the localization and tracking performance between the proposed distributed algorithm and the corresponding CADPD algorithm is negligible through simulation evaluations.
基金supported by the National Natural Science Foundation of China(No.61972261)Basic Research Foundations of Shenzhen(Nos.JCYJ 20210324093609026 and JCYJ20200813091134001).
文摘Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data increases at a pace that is faster than the increase in the big data processing capacity of clusters.Thus,distributed computing frameworks based on the MapReduce computing model are not adequate to support big data analysis tasks which often require running complex analytical algorithms on extremely big data sets in terabytes.In performing such tasks,these frameworks face three challenges:computational inefficiency due to high I/O and communication costs,non-scalability to big data due to memory limit,and limited analytical algorithms because many serial algorithms cannot be implemented in the MapReduce programming model.New distributed computing frameworks need to be developed to conquer these challenges.In this paper,we review MapReduce-type distributed computing frameworks that are currently used in handling big data and discuss their problems when conducting big data analysis.In addition,we present a non-MapReduce distributed computing framework that has the potential to overcome big data analysis challenges.
基金supported by the National Key Research and Development Program of China (2016YFB0900100)the State Key Program of National Natural Science Foundation of China (51537010)the project of State Grid Corporation of China (52110418000T)。
文摘To enhance the cost-effectiveness of bulk hybrid AC-DC power systems and promote wind consumption,this paper proposes a two-stage risk-based robust reserve scheduling(RRRS)model.Different from traditional robust optimization,the proposed model applies an adjustable uncertainty set rather than a fixed one.Thereby,the operational risk is optimized together with the dispatch schedules,with a reasonable admissible region of wind power obtained correspondingly.In addition,both the operational base point and adjustment capacity of tielines are optimized in the RRRS model,which enables reserve sharing among the connected areas to handle the significant wind uncertainties.Based on the alternating direction method of multipliers(ADMM),a fully distributed framework is presented to solve the RRRS model in a distributed way.A dynamic penalty factor adjustment strategy(DPA)is also developed and applied to enhance its convergence properties.Since only limited information needs to be exchanged during the solution process,the communication burden is reduced and regional information is protected.Case studies on the 2-area 12-bus system and 3-area 354-bus system illustrate the effectiveness of the proposed model and approach.