Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Regenerative Braking Energy Recovery System ofMetro Train Based on Dual-Mode PowerManagement

In order to fully utilize the regenerative braking energy of metro trains and stabilize the metro DC traction busbar voltage,a hybrid regenerative braking energy recovery system with a dual-mode power management strategy is proposed.Firstly,the construction of the hybrid regenerative braking energy recovery system is explained.Then,based on the power demand of low-voltage load in metro stations,a dual-mode power management strategy is proposed to allocate the reference power of each system according to the different working conditions,and the control methods of each system are set.Finally,the correctness and effectiveness of the dual-mode strategy are verified through simulation,and the proposed braking energy utilization scheme is compared with other singleform utilization schemes.The results illustrate that the hybrid system with the dual-mode strategy can effectively recycle the regenerative braking energy of metro train and inhibit the busbar voltage fluctuation;the proposed braking energy utilization scheme has certain advantages on energy recovery and DC bus voltage stabilization compared with other single-form schemes;the proposed power management strategy can correctly allocate the reference power of each system with a lower construction cost.

Enhancing Renewable Energy Integration:A Gaussian-Bare-Bones Levy Cheetah Optimization Approach to Optimal Power Flow in Electrical Networks

In the contemporary era,the global expansion of electrical grids is propelled by various renewable energy sources(RESs).Efficient integration of stochastic RESs and optimal power flow(OPF)management are critical for network optimization.This study introduces an innovative solution,the Gaussian Bare-Bones Levy Cheetah Optimizer(GBBLCO),addressing OPF challenges in power generation systems with stochastic RESs.The primary objective is to minimize the total operating costs of RESs,considering four functions:overall operating costs,voltage deviation management,emissions reduction,voltage stability index(VSI)and power loss mitigation.Additionally,a carbon tax is included in the objective function to reduce carbon emissions.Thorough scrutiny,using modified IEEE 30-bus and IEEE 118-bus systems,validates GBBLCO’s superior performance in achieving optimal solutions.Simulation results demonstrate GBBLCO’s efficacy in six optimization scenarios:total cost with valve point effects,total cost with emission and carbon tax,total cost with prohibited operating zones,active power loss optimization,voltage deviation optimization and enhancing voltage stability index(VSI).GBBLCO outperforms conventional techniques in each scenario,showcasing rapid convergence and superior solution quality.Notably,GBBLCO navigates complexities introduced by valve point effects,adapts to environmental constraints,optimizes costs while considering prohibited operating zones,minimizes active power losses,and optimizes voltage deviation by enhancing the voltage stability index(VSI)effectively.This research significantly contributes to advancing OPF,emphasizing GBBLCO’s improved global search capabilities and ability to address challenges related to local minima.GBBLCO emerges as a versatile and robust optimization tool for diverse challenges in power systems,offering a promising solution for the evolving needs of renewable energy-integrated power grids.

A Two-Layer Active Power Optimization and Coordinated Control for Regional Power Grid Partitioning to Promote Distributed Renewable Energy Consumption

With the large-scale development and utilization of renewable energy,industrial flexible loads,as a kind of loadside resource with strong regulation ability,provide new opportunities for the research on renewable energy consumption problem in power systems.This paper proposes a two-layer active power optimization model based on industrial flexible loads for power grid partitioning,aiming at improving the line over-limit problem caused by renewable energy consumption in power grids with high proportion of renewable energy,and achieving the safe,stable and economical operation of power grids.Firstly,according to the evaluation index of renewable energy consumption characteristics of line active power,the power grid is divided into several partitions,and the interzone tie lines are taken as the optimization objects.Then,on the basis of partitioning,a two-layer active power optimization model considering the power constraints of industrial flexible loads is established.The upper-layer model optimizes the planned power of the inter-zone tie lines under the constraint of the minimum peak-valley difference within a day;the lower-layer model optimizes the regional source-load dispatching plan of each resource in each partition under the constraint of theminimumoperation cost of the partition,so as to reduce the line overlimit phenomenon caused by renewable energy consumption and save the electricity cost of industrial flexible loads.Finally,through simulation experiments,it is verified that the proposed model can effectively mobilize industrial flexible loads to participate in power grid operation and improve the economic stability of power grid.

Application of Power Electronics Converters in Renewable Energy

Against the backdrop of global energy shortages and increasingly severe environmental pollution,renewable energy is gradually becoming a significant direction for future energy development.Power electronics converters,as the core technology for energy conversion and control,play a crucial role in enhancing the efficiency and stability of renewable energy systems.This paper explores the basic principles and functions of power electronics converters and their specific applications in photovoltaic power generation,wind power generation,and energy storage systems.Additionally,it analyzes the current innovations in high-efficiency energy conversion,multilevel conversion technology,and the application of new materials and devices.By studying these technologies,the aim is to promote the widespread application of power electronics converters in renewable energy systems and provide theoretical and technical support for achieving sustainable energy development.

Traction power systems for electrified railways:evolution,state of the art,and future trends

Traction power systems(TPSs)play a vital role in the operation of electrified railways.The transformation of conventional railway TPSs to novel structures is not only a trend to promote the development of electrified railways toward high-efficiency and resilience but also an inevitable requirement to achieve carbon neutrality target.On the basis of sorting out the power supply structures of conventional AC and DC modes,this paper first reviews the characteristics of the existing TPSs,such as weak power supply flexibility and low-energy efficiency.Furthermore,the power supply structures of various TPSs for future electrified railways are described in detail,which satisfy longer distance,low-carbon,high-efficiency,high-reliability and high-quality power supply requirements.Meanwhile,the application prospects of different traction modes are discussed from both technical and economic aspects.Eventually,this paper introduces the research progress of mixed-system electrified railways and traction power supply technologies without catenary system,speculates on the future development trends and challenges of TPSs and predicts that TPSs will be based on the continuous power supply mode,employing power electronic equipment and intelligent information technology to construct a railway comprehensive energy system with renewable energy.

Dynamic and Power Generation Features of A Wind-Wave Hybrid System Consisting of A Spar-Type Wind Turbine and An Annular Wave Energy Converter in Irregular Waves

Combining wave energy converters(WECs)with floating offshore wind turbines proves a potential strategy to achieve better use of marine renewable energy.The full coupling investigation on the dynamic and power generation features of the hybrid systems under operational sea states is necessary but limited by numerical simulation tools.Here an aero-hydro-servo-elastic coupling numerical tool is developed and applied to investigate the motion,mooring tension,and energy conversion performance of a hybrid system consisting of a spar-type floating wind turbine and an annular wave energy converter.Results show that the addition of the WEC has no significant negative effect on the dynamic performance of the platform and even enhances the rotational stability of the platform.For surge and pitch motion,the peak of the spectra is originated from the dominating wave component,whereas for the heave motion,the peak of the spectrum is the superposed effect of the dominating wave component and the resonance of the system.The addition of the annular WEC can slightly improve the wind power by making the rotor to be in a better position to face the incoming wind and provide considerable wave energy production,which can compensate for the downtime of the offshore wind.

Energy Management System with Power Offering Strategy for a Microgrid Integrated VPP

In the context of both the Virtual Power Plant (VPP) and microgrid(MG), the Energy Management System (EMS) is a key decision-maker forintegrating Distributed renewable Energy Resources (DERs) efficiently. TheEMS is regarded as a strong enabler of providing the optimized schedulingcontrol in operation and management of usage of disperse DERs and RenewableEnergy reSources (RES) such as a small-size wind-turbine (WT) andphotovoltaic (PV) energies. The main objective to be pursued by the EMSis the minimization of the overall operating cost of the MG integrated VPPnetwork. However, the minimization of the power peaks is a new objective andopen issue to a well-functional EMS, along with the maximization of profitin the energy market. Thus, both objectives have to be taken into accountat the same time. Thus, this paper proposes the EMS application incorporatingpower offering strategy applying a nature-inspired algorithm such asParticle Swarm Optimization (PSO) algorithm, in order to find the optimalsolution of the objective function in the context of the overall operating cost,the coordination of DERs, and the energy losses in a MG integrated VPPnetwork. For a fair DERs coordination with minimized power fluctuationsin the power flow, the power offering strategies with an active power controland re-distribution are proposed. Simulation results show that the proposedMG integrated VPP model with PSO-based EMS employing EgalitarianreDistribution (ED) power offering strategy is most feasible option for theoverall operating cost of VPP revenue. The total operating cost of the proposedEMS with ED strategy is 40.98$ compared to 432.8$ of MGs only withoutEMS. It is concluded that each MGs in the proposed VPP model intelligentlyparticipates in energy trading market compliant with the objective function,to minimize the overall cost and the power fluctuation.

Peer-to-Peer Energy Trading Method of Multi-Virtual Power Plants Based on Non-Cooperative Game

The current electricity market fails to consider the energy consumption characteristics of transaction subjects such as virtual power plants.Besides,the game relationship between transaction subjects needs to be further explored.This paper proposes a Peer-to-Peer energy trading method for multi-virtual power plants based on a non-cooperative game.Firstly,a coordinated control model of public buildings is incorporated into the scheduling framework of the virtual power plant,considering the energy consumption characteristics of users.Secondly,the utility functions of multiple virtual power plants are analyzed,and a non-cooperative game model is established to explore the game relationship between electricity sellers in the Peer-to-Peer transaction process.Finally,the influence of user energy consumption characteristics on the virtual power plant operation and the Peer-to-Peer transaction process is analyzed by case studies.Furthermore,the effect of different parameters on the Nash equilibrium point is explored,and the influence factors of Peer-to-Peer transactions between virtual power plants are summarized.According to the obtained results,compared with the central air conditioning set as constant temperature control strategy,the flexible control strategy proposed in this paper improves the market power of each VPP and the overall revenue of the VPPs.In addition,the upper limit of the service quotation of the market operator have a great impact on the transaction mode of VPPs.When the service quotation decreases gradually,the P2P transaction between VPPs is more likely to occur.

Identification of High-Risk Scenarios for Cascading Failures in New Energy Power Grids Based on Deep Embedding Clustering Algorithms

At present,the proportion of new energy in the power grid is increasing,and the random fluctuations in power output increase the risk of cascading failures in the power grid.In this paper,we propose a method for identifying high-risk scenarios of interlocking faults in new energy power grids based on a deep embedding clustering(DEC)algorithm and apply it in a risk assessment of cascading failures in different operating scenarios for new energy power grids.First,considering the real-time operation status and system structure of new energy power grids,the scenario cascading failure risk indicator is established.Based on this indicator,the risk of cascading failure is calculated for the scenario set,the scenarios are clustered based on the DEC algorithm,and the scenarios with the highest indicators are selected as the significant risk scenario set.The results of simulations with an example power grid show that our method can effectively identify scenarios with a high risk of cascading failures from a large number of scenarios.

Impact of interconnections and renewable energy integration on the Philippine-Sabah Power Grid systems

This study presents a comprehensive impact analysis of the rotor angle stability of a proposed international connection between the Philippines and Sabah,Malaysia,as part of the Association of Southeast Asian Nations(ASEAN)Power Grid.This study focuses on modeling and evaluating the dynamic performance of the interconnected system,considering the high penetration of renewable sources.Power flow,small signal stability,and transient stability analyses were conducted to assess the ability of the proposed linked power system models to withstand small and large disturbances,utilizing the Power Systems Analysis Toolbox(PSAT)software in MATLAB.All components used in the model are documented in the PSAT library.Currently,there is a lack of publicly available studies regarding the implementation of this specific system.Additionally,the study investigates the behavior of a system with a high penetration of renewable energy sources.Based on the findings,this study concludes that a system is generally stable when interconnection is realized,given its appropriate location and dynamic component parameters.Furthermore,the critical eigenvalues of the system also exhibited improvement as the renewable energy sources were augmented.

Research progress on protective coatings against molten nitrate salts for thermal energy storage in concentrating solar power plants

Concentrating solar power(CSP) has garnered considerable global attention as a reliable means of generating bulk electricity, effectively addressing the intermittent nature of solar resources.The integration of molten salt technology for thermal energy storage(TES) has further contributed to the growth of CSP plants;however, the corrosive nature of molten salts poses challenges to the durability of container materials, necessitating innovative corrosion mitigation strategies.This review summarizes scientific advancements in high-temperature anticorrosion coatings for molten nitrate salts, highlighting the key challenges and future trends.It also explores various coating types, including metallic, ceramic, and carbon-based coatings, and compares different coating deposition methods.This review emphasizes the need for durable coatings that meet long-term performance requirements and regulatory limitations, with an emphasis on carbon-based coatings and emerging nanomaterials.A combination of multiple coatings is required to achieve desirable anticorrosion properties while addressing material compatibility and cost considerations.The overall goal is to advance the manufacturing, assembly, and performance of CSP systems for increased efficiency, reliability, and durability in various applications.

Virtual Synchronous Generator Adaptive Control of Energy Storage Power Station Based on Physical Constraints

The virtual synchronous generator(VSG)can simulate synchronous machine’s operation mechanism in the control link of an energy storage converter,so that an electrochemical energy storage power station has the ability to actively support the power grid,from passive regulation to active support.Since energy storage is an important physical basis for realizing the inertia and damping characteristics in VSG control,energy storage constraints of the physical characteristics on the system control parameters are analyzed to provide a basis for the system parameter tuning.In a classic VSG control,its virtual inertia and damping coefficient remain unchanged.When the grid load changes greatly,the constant control strategy most likely result in the grid frequency deviation beyond the stable operation standard limitations.To solve this problem,a comprehensive control strategy considering electrified wire netting demand and energy storage unit state of charge(SOC)is proposed,and an adaptive optimization method of VSG parameters under different SOC is given.The energy storage battery can maintain a safe working state at any time and be smoothly disconnected,which can effectively improve the output frequency performance of energy storage system.Simulation results further demonstrated the effectiveness of the VSG control theoretical analysis.

Research on Equivalent Modeling Method of AC-DC Power Networks Integrating with Renewable Energy Generation

Along with the increasing integration of renewable energy generation in AC-DC power networks,investigating the dynamic behaviors of this complex system with a proper equivalent model is significant.This paper presents an equivalent modeling method for the AC-DC power networks with doubly-fed induction generator(DFIG)based wind farms to decrease the simulation scale and computational burden.For the AC-DC power networks,the equivalent modeling strategy in accordance with the physical structure simplification is stated.Regarding the DFIG-based wind farms,the equivalent modeling based on the sequential identification of multi-machine parameters using the improved chaotic cuckoo search algorithm(ICCSA)is conducted.In light of the MATLAB simulation platform,a two-zone four-DC interconnected power grid with wind farms is built to check the efficacy of the proposed equivalentmodelingmethod.Fromthe simulation analyses and comparative validation in different algorithms and cases,the proposed method can precisely reflect the steady and dynamic performance of the demonstrated system under N-1 and N-2 fault scenarios,and it can efficiently achieve the parameter identification of the wind farms and fulfill the equivalent modeling.Consequently,the proposed approach’s effectiveness and suitability are confirmed.

Directed Acyclic Graph Blockchain for Secure Spectrum Sharing and Energy Trading in Power IoT

Peer-to-peer(P2P)spectrum sharing and energy trading are promising solutions to locally satisfy spectrum and energy demands in power Internet of Things(IoT).However,implementation of largescale P2P spectrum sharing and energy trading confronts security and privacy challenges.In this paper,we exploit consortium blockchain and Directed Acyclic Graph(DAG)to propose a new secure and distributed spectrum sharing and energy trading framework in power IoT,named spectrum-energy chain,where a set of local aggregators(LAGs)cooperatively confirm the identity of the power devices by utilizing consortium blockchain,so as to form a main chain.Then,the local power devices verify spectrum and energy micro-transactions simultaneously but asynchronously to form local spectrum tangle and local energy tangle,respectively.Moreover,an iterative double auction based micro transactions scheme is designed to solve the spectrum and energy pricing and the amount of shared spectrum and energy among power devices.Security analysis and numerical results illustrate that the developed spectrum-energy chain and the designed iterative double auction based microtransactions scheme are secure and efficient for spectrum sharing and energy trading in power IoT.

Research on Coordinated Development and Optimization of Distribution Networks at All Levels in Distributed Power Energy Engineering

The uncertainty of distributed generation energy has dramatically challenged the coordinated development of distribution networks at all levels.This paper focuses on the multi-time-scale regulation model of distributed generation energy under normal conditions.The simulation results of the example verify the self-optimization characteristics and the effectiveness of real-time dispatching of the distribution network control technology at all levels under multiple time scales.

Joint UAV 3D deployment and sensor power allocation for energy-efficient and secure data collection

Unmanned aerial vehicles(UAVs) are advantageous for data collection in wireless sensor networks(WSNs) due to its low cost of use,flexible deployment,controllable mobility,etc. However,how to cope with the inherent issues of energy limitation and data security in the WSNs is challenging in such an application paradigm. To this end,based on the framework of physical layer security,an optimization problem for maximizing secrecy energy efficiency(EE) of data collection is formulated,which focuses on optimizing the UAV’s positions and the sensors’ transmit power. To overcome the difficulties in solving the optimization problem,the methods of fractional programming and successive convex approximation are then adopted to gradually transform the original problem into a series of tractable subproblems which are solved in an iterative manner. As shown in simulation results,by the joint designs in the spatial domain of UAV and the power domain of sensors,the proposed algorithm achieves a significant improvement of secrecy EE and rate.

Hybrid Power Bank Deployment Model for Energy Supply Coverage Optimization in Industrial Wireless Sensor Network

Energy supply is one of the most critical challenges of wireless sensor networks(WSNs)and industrial wireless sensor networks(IWSNs).While research on coverage optimization problem(COP)centers on the network’s monitoring coverage,this research focuses on the power banks’energy supply coverage.The study of 2-D and 3-D spaces is typical in IWSN,with the realistic environment being more complex with obstacles(i.e.,machines).A 3-D surface is the field of interest(FOI)in this work with the established hybrid power bank deployment model for the energy supply COP optimization of IWSN.The hybrid power bank deployment model is highly adaptive and flexible for new or existing plants already using the IWSN system.The model improves the power supply to a more considerable extent with the least number of power bank deployments.The main innovation in this work is the utilization of a more practical surface model with obstacles and training while improving the convergence speed and quality of the heuristic algorithm.An overall probabilistic coverage rate analysis of every point on the FOI is provided,not limiting the scope to target points or areas.Bresenham’s algorithm is extended from 2-D to 3-D surface to enhance the probabilistic covering model for coverage measurement.A dynamic search strategy(DSS)is proposed to modify the artificial bee colony(ABC)and balance the exploration and exploitation ability for better convergence toward eliminating NP-hard deployment problems.Further,the cellular automata(CA)is utilized to enhance the convergence speed.The case study based on two typical FOI in the IWSN shows that the CA scheme effectively speeds up the optimization process.Comparative experiments are conducted on four benchmark functions to validate the effectiveness of the proposed method.The experimental results show that the proposed algorithm outperforms the ABC and gbest-guided ABC(GABC)algorithms.The results show that the proposed energy coverage optimization method based on the hybrid power bank deployment model generates more accurate results than the results obtained by similar algorithms(i.e.,ABC,GABC).The proposed model is,therefore,effective and efficient for optimization in the IWSN.

A Tutorial Review of the Solar Power Curve: Regressions, Model Chains, and Their Hybridization and Probabilistic Extensions

Owing to the persisting hype in pushing toward global carbon neutrality,the study scope of atmospheric science is rapidly expanding.Among numerous trending topics,energy meteorology has been attracting the most attention hitherto.One essential skill of solar energy meteorologists is solar power curve modeling,which seeks to map irradiance and auxiliary weather variables to solar power,by statistical and/or physical means.In this regard,this tutorial review aims to deliver a complete overview of those fundamental scientific and engineering principles pertaining to the solar power curve.Solar power curves can be modeled in two primary ways,one of regression and the other of model chain.Both classes of modeling approaches,alongside their hybridization and probabilistic extensions,which allow accuracy improvement and uncertainty quantification,are scrutinized and contrasted thoroughly in this review.

A coordinated operation method of wind-PV-hydrogenstorage multi-agent energy system

Wind-photovoltaic(PV)-hydrogen-storage multi-agent energy systems are expected to play an important role in promoting renewable power utilization and decarbonization.In this study,a coordinated operation method was proposed for a wind-PVhydrogen-storage multi-agent energy system.First,a coordinated operation model was formulated for each agent considering peer-to-peer power trading.Second,a coordinated operation interactive framework for a multi-agent energy system was proposed based on the theory of the alternating direction method of multipliers.Third,a distributed interactive algorithm was proposed to protect the privacy of each agent and solve coordinated operation strategies.Finally,the effectiveness of the proposed coordinated operation method was tested on multi-agent energy systems with different structures,and the operational revenues of the wind power,PV,hydrogen,and energy storage agents of the proposed coordinated operation model were improved by approximately 59.19%,233.28%,16.75%,and 145.56%,respectively,compared with the independent operation model.