In several fields like financial dealing,industry,business,medicine,et cetera,Big Data(BD)has been utilized extensively,which is nothing but a collection of a huge amount of data.However,it is highly complicated alon...In several fields like financial dealing,industry,business,medicine,et cetera,Big Data(BD)has been utilized extensively,which is nothing but a collection of a huge amount of data.However,it is highly complicated along with time-consuming to process a massive amount of data.Thus,to design the Distribution Preserving Framework for BD,a novel methodology has been proposed utilizing Manhattan Distance(MD)-centered Partition Around Medoid(MD–PAM)along with Conjugate Gradient Artificial Neural Network(CG-ANN),which undergoes various steps to reduce the complications of BD.Firstly,the data are processed in the pre-processing phase by mitigating the data repetition utilizing the map-reduce function;subsequently,the missing data are handled by substituting or by ignoring the missed values.After that,the data are transmuted into a normalized form.Next,to enhance the classification performance,the data’s dimensionalities are minimized by employing Gaussian Kernel(GK)-Fisher Discriminant Analysis(GK-FDA).Afterwards,the processed data is submitted to the partitioning phase after transmuting it into a structured format.In the partition phase,by utilizing the MD-PAM,the data are partitioned along with grouped into a cluster.Lastly,by employing CG-ANN,the data are classified in the classification phase so that the needed data can be effortlessly retrieved by the user.To analogize the outcomes of the CG-ANN with the prevailing methodologies,the NSL-KDD openly accessible datasets are utilized.The experiential outcomes displayed that an efficient result along with a reduced computation cost was shown by the proposed CG-ANN.The proposed work outperforms well in terms of accuracy,sensitivity and specificity than the existing systems.展开更多
The conventional direct position determination(DPD) algorithm processes all received signals on a single sensor.When sensors have limited computational capabilities or energy storage,it is desirable to distribute th...The conventional direct position determination(DPD) algorithm processes all received signals on a single sensor.When sensors have limited computational capabilities or energy storage,it is desirable to distribute the computation among other sensors.A distributed adaptive DPD(DADPD)algorithm based on diffusion framework is proposed for emitter localization.Unlike the corresponding centralized adaptive DPD(CADPD) algorithm,all but one sensor in the proposed algorithm participate in processing the received signals and estimating the common emitter position,respectively.The computational load and energy consumption on a single sensor in the CADPD algorithm is distributed among other computing sensors in a balanced manner.Exactly the same iterative localization algorithm is carried out in each computing sensor,respectively,and the algorithm in each computing sensor exhibits quite similar convergence behavior.The difference of the localization and tracking performance between the proposed distributed algorithm and the corresponding CADPD algorithm is negligible through simulation evaluations.展开更多
When data privacy is imposed as a necessity,Federated learning(FL)emerges as a relevant artificial intelligence field for developing machine learning(ML)models in a distributed and decentralized environment.FL allows ...When data privacy is imposed as a necessity,Federated learning(FL)emerges as a relevant artificial intelligence field for developing machine learning(ML)models in a distributed and decentralized environment.FL allows ML models to be trained on local devices without any need for centralized data transfer,thereby reducing both the exposure of sensitive data and the possibility of data interception by malicious third parties.This paradigm has gained momentum in the last few years,spurred by the plethora of real-world applications that have leveraged its ability to improve the efficiency of distributed learning and to accommodate numerous participants with their data sources.By virtue of FL,models can be learned from all such distributed data sources while preserving data privacy.The aim of this paper is to provide a practical tutorial on FL,including a short methodology and a systematic analysis of existing software frameworks.Furthermore,our tutorial provides exemplary cases of study from three complementary perspectives:i)Foundations of FL,describing the main components of FL,from key elements to FL categories;ii)Implementation guidelines and exemplary cases of study,by systematically examining the functionalities provided by existing software frameworks for FL deployment,devising a methodology to design a FL scenario,and providing exemplary cases of study with source code for different ML approaches;and iii)Trends,shortly reviewing a non-exhaustive list of research directions that are under active investigation in the current FL landscape.The ultimate purpose of this work is to establish itself as a referential work for researchers,developers,and data scientists willing to explore the capabilities of FL in practical applications.展开更多
This paper develops a fully distributed hybrid control framework for distributed constrained optimization problems.The individual cost functions are non-differentiable and convex.Based on hybrid dynamical systems,we p...This paper develops a fully distributed hybrid control framework for distributed constrained optimization problems.The individual cost functions are non-differentiable and convex.Based on hybrid dynamical systems,we present a distributed state-dependent hybrid design to improve the transient performance of distributed primal-dual first-order optimization methods.The proposed framework consists of a distributed constrained continuous-time mapping in the form of a differential inclusion and a distributed discrete-time mapping triggered by the satisfaction of local jump set.With the semistability theory of hybrid dynamical systems,the paper proves that the hybrid control algorithm converges to one optimal solution instead of oscillating among different solutions.Numerical simulations illustrate better transient performance of the proposed hybrid algorithm compared with the results of the existing continuous-time algorithms.展开更多
The distributed denial of service (DDoS) attack is one of the dangers in intrusion modes. It's difficult to defense and can cause serious damage to the system. Based on a careful study of the attack principles and...The distributed denial of service (DDoS) attack is one of the dangers in intrusion modes. It's difficult to defense and can cause serious damage to the system. Based on a careful study of the attack principles and characteristics, an object-oriented formalized description is presented, which contains a three-level framework and offers full specifications of all kinds of DDoS modes and their features and the relations between one another. Its greatest merit lies in that it contributes to analyzing, checking and judging DDoS. Now this formalized description has been used in a special IDS and it works very effectively.(展开更多
In distribution systems,network reconfiguration and capacitor placement are commonly used to diminish power losses and keep voltage profiles within acceptable limits.Moreover,the problem of DG allocation and sizing is...In distribution systems,network reconfiguration and capacitor placement are commonly used to diminish power losses and keep voltage profiles within acceptable limits.Moreover,the problem of DG allocation and sizing is great important.In this work,a combination of a fuzzy multi-objective approach and bacterial foraging optimization(BFO) as a meta-heuristic algorithm is used to solve the simultaneous reconfiguration and optimal sizing of DGs and shunt capacitors in a distribution system.Each objective is transferred into fuzzy domain using its membership function.Then,the overall fuzzy satisfaction function is formed and considered a fitness function inasmuch as the value of this function has to be maximized to gain the optimal solution.The numerical results show that the presented algorithm improves the performance much more than other meta-heuristic algorithms.Simulation results found that simultaneous reconfiguration with DG and shunt capacitors allocation(case 5) has 77.41%,42.15%,and 56.14%improvements in power loss reduction,load balancing,and voltage profile indices,respectively in 33-bus test system.This result found 87.27%,35.82%,and 54.34%improvements of mentioned indices respectively for 69-bus system.展开更多
针对配电站房缺乏健康评估机制、运维周期设置不合理的问题,提出了一种考虑群体决策差异冲突解决机制的配电站房健康状态综合评估方法。首先,建立配电站房指标体系和专家评价框架,设计了一种新型的二元冲突测量函数来量化全局冲突。然后...针对配电站房缺乏健康评估机制、运维周期设置不合理的问题,提出了一种考虑群体决策差异冲突解决机制的配电站房健康状态综合评估方法。首先,建立配电站房指标体系和专家评价框架,设计了一种新型的二元冲突测量函数来量化全局冲突。然后,使用专家评价结果的虚假度、可信度、可用度等测度指标构造专家修正因子,以改进D-S证据理论,通过聚合不同专家的评价意见来量化评价指标的权重。接着,建立改进灰色关联度-逼近理想解法(grey relation analysis-technique for order preference by similarity to an ideal solution, GRA-TOPSIS)评估模型,引入灰色关联接近度,与距离接近度融合得到综合接近度,改善TOPSIS评价判据片面性的缺陷。最后,计算每个配电站房的评价值与理想解之间的综合接近度,反映配电站房的健康状态。实验分析表明该方法能兼容专家评价之间的冲突性、差异性、不确定性,与现有方法相比评估结果更具准确性和合理性,对运维人员制定合理的检修决策具有一定的指导价值。展开更多
With the development of green data centers,a large number of Uninterruptible Power Supply(UPS)resources in Internet Data Center(IDC)are becoming idle assets owing to their low utilization rate.The revitalization of th...With the development of green data centers,a large number of Uninterruptible Power Supply(UPS)resources in Internet Data Center(IDC)are becoming idle assets owing to their low utilization rate.The revitalization of these idle UPS resources is an urgent problem that must be addressed.Based on the energy storage type of the UPS(EUPS)and using renewable sources,a solution for IDCs is proposed in this study.Subsequently,an EUPS cluster classification method based on the concept of shared mechanism niche(CSMN)was proposed to effectively solve the EUPS control problem.Accordingly,the classified EUPS aggregation unit was used to determine the optimal operation of the IDC.An IDC cost minimization optimization model was established,and the Quantum Particle Swarm Optimization(QPSO)algorithm was adopted.Finally,the economy and effectiveness of the three-tier optimization framework and model were verified through three case studies.展开更多
The increasing use of renewable energy in the power system results in strong stochastic disturbances and degrades the control performance of the distributed power grids.In this paper,a novel multi-agent collaborative ...The increasing use of renewable energy in the power system results in strong stochastic disturbances and degrades the control performance of the distributed power grids.In this paper,a novel multi-agent collaborative reinforcement learning algorithm is proposed with automatic optimization,namely,Dyna-DQL,to quickly achieve an optimal coordination solution for the multi-area distributed power grids.The proposed Dyna framework is combined with double Q-learning to collect and store the environmental samples.This can iteratively update the agents through buffer replay and real-time data.Thus the environmental data can be fully used to enhance the learning speed of the agents.This mitigates the negative impact of heavy stochastic disturbances caused by the integration of renewable energy on the control performance.Simulations are conducted on two different models to validate the effectiveness of the proposed algorithm.The results demonstrate that the proposed Dyna-DQL algorithm exhibits superior stability and robustness compared to other reinforcement learning algorithms.展开更多
Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data i...Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data increases at a pace that is faster than the increase in the big data processing capacity of clusters.Thus,distributed computing frameworks based on the MapReduce computing model are not adequate to support big data analysis tasks which often require running complex analytical algorithms on extremely big data sets in terabytes.In performing such tasks,these frameworks face three challenges:computational inefficiency due to high I/O and communication costs,non-scalability to big data due to memory limit,and limited analytical algorithms because many serial algorithms cannot be implemented in the MapReduce programming model.New distributed computing frameworks need to be developed to conquer these challenges.In this paper,we review MapReduce-type distributed computing frameworks that are currently used in handling big data and discuss their problems when conducting big data analysis.In addition,we present a non-MapReduce distributed computing framework that has the potential to overcome big data analysis challenges.展开更多
文摘In several fields like financial dealing,industry,business,medicine,et cetera,Big Data(BD)has been utilized extensively,which is nothing but a collection of a huge amount of data.However,it is highly complicated along with time-consuming to process a massive amount of data.Thus,to design the Distribution Preserving Framework for BD,a novel methodology has been proposed utilizing Manhattan Distance(MD)-centered Partition Around Medoid(MD–PAM)along with Conjugate Gradient Artificial Neural Network(CG-ANN),which undergoes various steps to reduce the complications of BD.Firstly,the data are processed in the pre-processing phase by mitigating the data repetition utilizing the map-reduce function;subsequently,the missing data are handled by substituting or by ignoring the missed values.After that,the data are transmuted into a normalized form.Next,to enhance the classification performance,the data’s dimensionalities are minimized by employing Gaussian Kernel(GK)-Fisher Discriminant Analysis(GK-FDA).Afterwards,the processed data is submitted to the partitioning phase after transmuting it into a structured format.In the partition phase,by utilizing the MD-PAM,the data are partitioned along with grouped into a cluster.Lastly,by employing CG-ANN,the data are classified in the classification phase so that the needed data can be effortlessly retrieved by the user.To analogize the outcomes of the CG-ANN with the prevailing methodologies,the NSL-KDD openly accessible datasets are utilized.The experiential outcomes displayed that an efficient result along with a reduced computation cost was shown by the proposed CG-ANN.The proposed work outperforms well in terms of accuracy,sensitivity and specificity than the existing systems.
基金supported by the National Natural Science Foundation of China(61101173)
文摘The conventional direct position determination(DPD) algorithm processes all received signals on a single sensor.When sensors have limited computational capabilities or energy storage,it is desirable to distribute the computation among other sensors.A distributed adaptive DPD(DADPD)algorithm based on diffusion framework is proposed for emitter localization.Unlike the corresponding centralized adaptive DPD(CADPD) algorithm,all but one sensor in the proposed algorithm participate in processing the received signals and estimating the common emitter position,respectively.The computational load and energy consumption on a single sensor in the CADPD algorithm is distributed among other computing sensors in a balanced manner.Exactly the same iterative localization algorithm is carried out in each computing sensor,respectively,and the algorithm in each computing sensor exhibits quite similar convergence behavior.The difference of the localization and tracking performance between the proposed distributed algorithm and the corresponding CADPD algorithm is negligible through simulation evaluations.
基金the R&D&I,Spain grants PID2020-119478GB-I00 and,PID2020-115832GB-I00 funded by MCIN/AEI/10.13039/501100011033.N.Rodríguez-Barroso was supported by the grant FPU18/04475 funded by MCIN/AEI/10.13039/501100011033 and by“ESF Investing in your future”Spain.J.Moyano was supported by a postdoctoral Juan de la Cierva Formación grant FJC2020-043823-I funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR.J.Del Ser acknowledges funding support from the Spanish Centro para el Desarrollo Tecnológico Industrial(CDTI)through the AI4ES projectthe Department of Education of the Basque Government(consolidated research group MATHMODE,IT1456-22)。
文摘When data privacy is imposed as a necessity,Federated learning(FL)emerges as a relevant artificial intelligence field for developing machine learning(ML)models in a distributed and decentralized environment.FL allows ML models to be trained on local devices without any need for centralized data transfer,thereby reducing both the exposure of sensitive data and the possibility of data interception by malicious third parties.This paradigm has gained momentum in the last few years,spurred by the plethora of real-world applications that have leveraged its ability to improve the efficiency of distributed learning and to accommodate numerous participants with their data sources.By virtue of FL,models can be learned from all such distributed data sources while preserving data privacy.The aim of this paper is to provide a practical tutorial on FL,including a short methodology and a systematic analysis of existing software frameworks.Furthermore,our tutorial provides exemplary cases of study from three complementary perspectives:i)Foundations of FL,describing the main components of FL,from key elements to FL categories;ii)Implementation guidelines and exemplary cases of study,by systematically examining the functionalities provided by existing software frameworks for FL deployment,devising a methodology to design a FL scenario,and providing exemplary cases of study with source code for different ML approaches;and iii)Trends,shortly reviewing a non-exhaustive list of research directions that are under active investigation in the current FL landscape.The ultimate purpose of this work is to establish itself as a referential work for researchers,developers,and data scientists willing to explore the capabilities of FL in practical applications.
基金supported in part by the NationalKey Research and Development Program of China(2021YFB1714800)the National Natural Science Foundation of China(61925303,62088101,62073035,62173034)the Natural Science Foundation of Chongqing(2021ZX4100027)。
文摘This paper develops a fully distributed hybrid control framework for distributed constrained optimization problems.The individual cost functions are non-differentiable and convex.Based on hybrid dynamical systems,we present a distributed state-dependent hybrid design to improve the transient performance of distributed primal-dual first-order optimization methods.The proposed framework consists of a distributed constrained continuous-time mapping in the form of a differential inclusion and a distributed discrete-time mapping triggered by the satisfaction of local jump set.With the semistability theory of hybrid dynamical systems,the paper proves that the hybrid control algorithm converges to one optimal solution instead of oscillating among different solutions.Numerical simulations illustrate better transient performance of the proposed hybrid algorithm compared with the results of the existing continuous-time algorithms.
文摘The distributed denial of service (DDoS) attack is one of the dangers in intrusion modes. It's difficult to defense and can cause serious damage to the system. Based on a careful study of the attack principles and characteristics, an object-oriented formalized description is presented, which contains a three-level framework and offers full specifications of all kinds of DDoS modes and their features and the relations between one another. Its greatest merit lies in that it contributes to analyzing, checking and judging DDoS. Now this formalized description has been used in a special IDS and it works very effectively.(
文摘In distribution systems,network reconfiguration and capacitor placement are commonly used to diminish power losses and keep voltage profiles within acceptable limits.Moreover,the problem of DG allocation and sizing is great important.In this work,a combination of a fuzzy multi-objective approach and bacterial foraging optimization(BFO) as a meta-heuristic algorithm is used to solve the simultaneous reconfiguration and optimal sizing of DGs and shunt capacitors in a distribution system.Each objective is transferred into fuzzy domain using its membership function.Then,the overall fuzzy satisfaction function is formed and considered a fitness function inasmuch as the value of this function has to be maximized to gain the optimal solution.The numerical results show that the presented algorithm improves the performance much more than other meta-heuristic algorithms.Simulation results found that simultaneous reconfiguration with DG and shunt capacitors allocation(case 5) has 77.41%,42.15%,and 56.14%improvements in power loss reduction,load balancing,and voltage profile indices,respectively in 33-bus test system.This result found 87.27%,35.82%,and 54.34%improvements of mentioned indices respectively for 69-bus system.
文摘针对配电站房缺乏健康评估机制、运维周期设置不合理的问题,提出了一种考虑群体决策差异冲突解决机制的配电站房健康状态综合评估方法。首先,建立配电站房指标体系和专家评价框架,设计了一种新型的二元冲突测量函数来量化全局冲突。然后,使用专家评价结果的虚假度、可信度、可用度等测度指标构造专家修正因子,以改进D-S证据理论,通过聚合不同专家的评价意见来量化评价指标的权重。接着,建立改进灰色关联度-逼近理想解法(grey relation analysis-technique for order preference by similarity to an ideal solution, GRA-TOPSIS)评估模型,引入灰色关联接近度,与距离接近度融合得到综合接近度,改善TOPSIS评价判据片面性的缺陷。最后,计算每个配电站房的评价值与理想解之间的综合接近度,反映配电站房的健康状态。实验分析表明该方法能兼容专家评价之间的冲突性、差异性、不确定性,与现有方法相比评估结果更具准确性和合理性,对运维人员制定合理的检修决策具有一定的指导价值。
基金supported by the Key Technology Projects of the China Southern Power Grid Corporation(STKJXM20200059)the Key Support Project of the Joint Fund of the National Natural Science Foundation of China(U22B20123)。
文摘With the development of green data centers,a large number of Uninterruptible Power Supply(UPS)resources in Internet Data Center(IDC)are becoming idle assets owing to their low utilization rate.The revitalization of these idle UPS resources is an urgent problem that must be addressed.Based on the energy storage type of the UPS(EUPS)and using renewable sources,a solution for IDCs is proposed in this study.Subsequently,an EUPS cluster classification method based on the concept of shared mechanism niche(CSMN)was proposed to effectively solve the EUPS control problem.Accordingly,the classified EUPS aggregation unit was used to determine the optimal operation of the IDC.An IDC cost minimization optimization model was established,and the Quantum Particle Swarm Optimization(QPSO)algorithm was adopted.Finally,the economy and effectiveness of the three-tier optimization framework and model were verified through three case studies.
基金supported by the National Natural Sci-ence Foundation of China(No.52277108)Guangdong Provincial Department of Science and Technology(No.2022A0505020015).
文摘The increasing use of renewable energy in the power system results in strong stochastic disturbances and degrades the control performance of the distributed power grids.In this paper,a novel multi-agent collaborative reinforcement learning algorithm is proposed with automatic optimization,namely,Dyna-DQL,to quickly achieve an optimal coordination solution for the multi-area distributed power grids.The proposed Dyna framework is combined with double Q-learning to collect and store the environmental samples.This can iteratively update the agents through buffer replay and real-time data.Thus the environmental data can be fully used to enhance the learning speed of the agents.This mitigates the negative impact of heavy stochastic disturbances caused by the integration of renewable energy on the control performance.Simulations are conducted on two different models to validate the effectiveness of the proposed algorithm.The results demonstrate that the proposed Dyna-DQL algorithm exhibits superior stability and robustness compared to other reinforcement learning algorithms.
基金supported by the National Natural Science Foundation of China(No.61972261)Basic Research Foundations of Shenzhen(Nos.JCYJ 20210324093609026 and JCYJ20200813091134001).
文摘Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data increases at a pace that is faster than the increase in the big data processing capacity of clusters.Thus,distributed computing frameworks based on the MapReduce computing model are not adequate to support big data analysis tasks which often require running complex analytical algorithms on extremely big data sets in terabytes.In performing such tasks,these frameworks face three challenges:computational inefficiency due to high I/O and communication costs,non-scalability to big data due to memory limit,and limited analytical algorithms because many serial algorithms cannot be implemented in the MapReduce programming model.New distributed computing frameworks need to be developed to conquer these challenges.In this paper,we review MapReduce-type distributed computing frameworks that are currently used in handling big data and discuss their problems when conducting big data analysis.In addition,we present a non-MapReduce distributed computing framework that has the potential to overcome big data analysis challenges.