Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Regenerative Braking Energy Recovery System of Metro Train Based on Dual-Mode Power Management

In order to fully utilize the regenerative braking energy of metro trains and stabilize the metro DC traction busbar voltage,a hybrid regenerative braking energy recovery system with a dual-mode power management strategy is proposed.Firstly,the construction of the hybrid regenerative braking energy recovery system is explained.Then,based on the power demand of low-voltage load in metro stations,a dual-mode power management strategy is proposed to allocate the reference power of each system according to the different working conditions,and the control methods of each system are set.Finally,the correctness and effectiveness of the dual-mode strategy are verified through simulation,and the proposed braking energy utilization scheme is compared with other singleform utilization schemes.The results illustrate that the hybrid system with the dual-mode strategy can effectively recycle the regenerative braking energy of metro train and inhibit the busbar voltage fluctuation;the proposed braking energy utilization scheme has certain advantages on energy recovery and DC bus voltage stabilization compared with other single-form schemes;the proposed power management strategy can correctly allocate the reference power of each system with a lower construction cost.

Smart Energy Management System Using Machine Learning

Energy management is an inspiring domain in developing of renewable energy sources.However,the growth of decentralized energy production is revealing an increased complexity for power grid managers,inferring more quality and reliability to regulate electricity flows and less imbalance between electricity production and demand.The major objective of an energy management system is to achieve optimum energy procurement and utilization throughout the organization,minimize energy costs without affecting production,and minimize environmental effects.Modern energy management is an essential and complex subject because of the excessive consumption in residential buildings,which necessitates energy optimization and increased user comfort.To address the issue of energy management,many researchers have developed various frameworks;while the objective of each framework was to sustain a balance between user comfort and energy consumption,this problem hasn’t been fully solved because of how difficult it is to solve it.An inclusive and Intelligent Energy Management System(IEMS)aims to provide overall energy efficiency regarding increased power generation,increase flexibility,increase renewable generation systems,improve energy consumption,reduce carbon dioxide emissions,improve stability,and reduce energy costs.Machine Learning(ML)is an emerging approach that may be beneficial to predict energy efficiency in a better way with the assistance of the Internet of Energy(IoE)network.The IoE network is playing a vital role in the energy sector for collecting effective data and usage,resulting in smart resource management.In this research work,an IEMS is proposed for Smart Cities(SC)using the ML technique to better resolve the energy management problem.The proposed system minimized the energy consumption with its intelligent nature and provided better outcomes than the previous approaches in terms of 92.11% accuracy,and 7.89% miss-rate.

Reinforcement Learning Model for Energy System Management to Ensure Energy Efficiency and Comfort in Buildings

This article focuses on the challenges ofmodeling energy supply systems for buildings,encompassing both methods and tools for simulating thermal regimes and engineering systems within buildings.Enhancing the comfort of living or working in buildings often necessitates increased consumption of energy and material,such as for thermal upgrades,which consequently incurs additional economic costs.It is crucial to acknowledge that such improvements do not always lead to a decrease in total pollutant emissions,considering emissions across all stages of production and usage of energy and materials aimed at boosting energy efficiency and comfort in buildings.In addition,it explores the methods and mechanisms for modeling the operating modes of electric boilers used to collectively improve energy efficiency and indoor climatic conditions.Using the developed mathematical models,the study examines the dynamic states of building energy supply systems and provides recommendations for improving their efficiency.These dynamic models are executed in software environments such as MATLAB/Simscape and Python,where the component detailing schemes for various types of controllers are demonstrated.Additionally,controllers based on reinforcement learning(RL)displayed more adaptive load level management.These RL-based controllers can lower instantaneous power usage by up to 35%,reduce absolute deviations from a comfortable temperature nearly by half,and cut down energy consumption by approximately 1%while maintaining comfort.When the energy source produces a constant energy amount,the RL-based heat controllermore effectively maintains the temperature within the set range,preventing overheating.In conclusion,the introduced energydynamic building model and its software implementation offer a versatile tool for researchers,enabling the simulation of various energy supply systems to achieve optimal energy efficiency and indoor climate control in buildings.

Research on Smart Energy Monitoring and Management System Based on Digital Twin Technology

Smart energy monitoring and management system lays a foundation for the application and development of smart energy. However, in recent years, the work efficiency of smart energy development enterprises has generally been low, and there is an urgent need to improve the application efficiency, resilience and sustainability of smart energy monitoring and management system. Digital twin technology provides a data-centric solution to improve smart energy monitoring and management system, bringing an opportunity to transform passive infrastructure assets into data-centric systems. This paper expounds on the concept and key technologies of digital twin, and designs a smart energy monitoring and management system based on digital twin technology, which has dual significance for promoting the development of smart energy field and promoting the application of digital twin.

Survey on AI and Machine Learning Techniques for Microgrid Energy Management Systems

In the era of an energy revolution,grid decentralization has emerged as a viable solution to meet the increasing global energy demand by incorporating renewables at the distributed level.Microgrids are considered a driving component for accelerating grid decentralization.To optimally utilize the available resources and address potential challenges,there is a need to have an intelligent and reliable energy management system(EMS)for the microgrid.The artificial intelligence field has the potential to address the problems in EMS and can provide resilient,efficient,reliable,and scalable solutions.This paper presents an overview of existing conventional and AI-based techniques for energy management systems in microgrids.We analyze EMS methods for centralized,decentralized,and distributed microgrids separately.Then,we summarize machine learning techniques such as ANNs,federated learning,LSTMs,RNNs,and reinforcement learning for EMS objectives such as economic dispatch,optimal power flow,and scheduling.With the incorporation of AI,microgrids can achieve greater performance efficiency and more reliability for managing a large number of energy resources.However,challenges such as data privacy,security,scalability,explainability,etc.,need to be addressed.To conclude,the authors state the possible future research directions to explore AI-based EMS's potential in real-world applications.

Sustainable Energy Management with Traffic Prediction Strategy for Autonomous Vehicle Systems

Recent advancements of the intelligent transportation system(ITS)provide an effective way of improving the overall efficiency of the energy management strategy(EMSs)for autonomous vehicles(AVs).The use of AVs possesses many advantages such as congestion control,accident prevention,and etc.However,energy management and traffic flow prediction(TFP)still remains a challenging problem in AVs.The complexity and uncertainties of driving situations adequately affect the outcome of the designed EMSs.In this view,this paper presents novel sustainable energy management with traffic flow prediction strategy(SEM-TPS)for AVs.The SEM-TPS technique applies type II fuzzy logic system(T2FLS)energy management scheme to accomplish the desired engine torque based on distinct parameters.In addition,the membership functions of the T2FLS scheme are chosen optimally using the barnacles mating optimizer(BMO).For accurate TFP,the bidirectional gated recurrent neural network(Bi-GRNN)model is used in AVs.A comprehensive experimental validation process is performed and the results are inspected with respect to several evaluation metrics.The experimental outcomes highlighted the supreme performance of the SEM-TPS technique over the recent state of art approaches.

Energy Efficiency in Smart Grid: A Prospective Study on Energy Management Systems

The term Smart Grid has become a term to represent the benefits of a smart and sophisticated electrical grid, which can meet various social expectations related to sustainability and energy efficiency. The Smart Grid promises to enable a better power management for energy utilities and consumers, to provide the ability to integrate the power grid, to support the development of micro grids, and to involve citizens in energy management with higher levels of responsibility. However, this context comes with potential pitfalls, such as vulnerabilities to cyber-security and privacy risks. In this article, a prospective study about energy management, and exploring critical issues of modeling of energy management systems in a context Smart. Grid is presented along with background of energy management systems. An analysis of the demand response condition is also presented. Finally, the advantages and disadvantages of the implementation of energy management systems, and a comparison with the Brazilian electricity system are discussed.

Autonomous Multi-Factor Energy Flows Controller (AmEFC): Enhancing Renewable Energy Management with Intelligent Control Systems Integration

The transition to sustainable energy systems is one of the defining challenges of our time, necessitating innovations in how we generate, distribute, and manage electrical power. Micro-grids, as localized energy hubs, have emerged as a promising solution to integrate renewable energy sources, ensure energy security, and improve system resilience. The Autonomous multi-factor Energy Flow Controller (AmEFC) introduced in this paper addresses this need by offering a scalable, adaptable, and resilient framework for energy management within an on-grid micro-grid context. The urgency for such a system is predicated on the increasing volatility and unpredictability in energy landscapes, including fluctuating renewable outputs and changing load demands. To tackle these challenges, the AmEFC prototype incorporates a novel hierarchical control structure that leverages Renewable Energy Sources (RES), such as photovoltaic systems, wind turbines, and hydro pumps, alongside a sophisticated Battery Management System (BMS). Its prime objective is to maintain an uninterrupted power supply to critical loads, efficiently balance energy surplus through hydraulic storage, and ensure robust interaction with the main grid. A comprehensive Simulink model is developed to validate the functionality of the AmEFC, simulating real-world conditions and dynamic interactions among the components. The model assesses the system’s reliability in consistently powering critical loads and its efficacy in managing surplus energy. The inclusion of advanced predictive algorithms enables the AmEFC to anticipate energy production and consumption trends, integrating weather forecasting and inter-controller communication to optimize energy flow within and across micro-grids. This study’s significance lies in its potential to facilitate the seamless incorporation of RES into existing power systems, thus propelling the energy sector towards a more sustainable, autonomous, and resilient future. The results underscore the potential of such a system to revolutionize energy management practices and highlight the importance of smart controller systems in the era of smart grids.

Comparison of Energy Harvesting Systems for Wireless Sensor Networks

Wireless sensor networks (WSNs) offer an attractive solution to many environmental,security,and process monitoring problems.However,one barrier to their fuller adoption is the need to supply electrical power over extended periods of time without the need for dedicated wiring.Energy harvesting provides a potential solution to this problem in many applications.This paper reviews the characteristics and energy requirements of typical sensor network nodes,assesses a range of potential ambient energy sources,and outlines the characteristics of a wide range of energy conversion devices.It then proposes a method to compare these diverse sources and conversion mechanisms in terms of their normalised power density.

Prospects of key technologies of integrated energy systems for rural electrification in China

Owing to increasing environmental concerns and resource scarcity, integrated energy system shave become widely used in communities. Rural energy systems, as one of the important links of the energy network in China, suffer from low energy efficiency and weak infrastructure. Therefore, it is particularly important to increase the proportion of electricity consumption and build an integrated energy system for rural electrification in China(IESREIC) with a rural distribution network as the core, in line with national conditions. In this study, by analyzing the Chinese regional differences and natural resource endowments, the development characteristics of the IESREIC are summarized. Then, according to the existing rural energy problems, key technologies are proposed for the IESREIC, such as those for planning and operation, value sharing, infrastructure, and a management and control platform. Finally, IESREIC demonstration projects and business models are introduced for agricultural production, rural industrial systems, and rural life. The purpose is to propose research concepts for the IESREIC, provide suggestions for the development of rural energy, and provide a reference for the construction of rural energy systems in countries with characteristics similar to those of China.

Blockchain Energy: Blockchain in Future Energy Systems

The ongoing,in-depth transformation of the electricity sector towards increased use of alternative,renewable energy sources extends beyond a simple decentralisation drive in the electricity market.The transformation process is characterised by the interplay of old and new technologies from the energy sector as well as structural coupling with other sectors,such as the information and communications technology(ICT),enabling the technology transfer as well as market entry by information technology(IT)actors.Blockchain-based technologies have the potential to play a key role in this transition by offering decentralised interfaces and systems as well as an alternative approach to the current organisation form of the energy market.This paper discusses the applicability and prospects for blockchain-based technologies in the energy sector,which are described using the term“blockchain energy”.For the purposes of this study,blockchain energy encompasses all socio-technical and organisational configurations in the energy sector based on the utilisation of the blockchain principle for energy trading,information storage,and/or increased transparency of energy flows and energy services.In the following chapters,the authors present and discuss the current transformation in the electricity market,followed by a review of the different utilisation possibilities for blockchain technologies in the energy sector and a discussion of the barriers and potential for blockchain energy using a transition studies perspective.Finally,the opportunities and risks of blockchain energy are discussed.

Applications of Lithium-Ion Batteries in Grid-Scale Energy Storage Systems

In the electrical energy transformation process,the grid-level energy storage system plays an essential role in balancing power generation and utilization.Batteries have considerable potential for application to grid-level energy storage systems because of their rapid response,modularization,and flexible installation.Among several battery technologies,lithium-ion batteries(LIBs)exhibit high energy efficiency,long cycle life,and relatively high energy density.In this perspective,the properties of LIBs,including their operation mechanism,battery design and construction,and advantages and disadvantages,have been analyzed in detail.Moreover,the performance of LIBs applied to grid-level energy storage systems is analyzed in terms of the following grid services:(1)frequency regulation;(2)peak shifting;(3)integration with renewable energy sources;and(4)power management.In addition,the challenges encountered in the application of LIBs are discussed and possible research directions aimed at overcoming these challenges are proposed to provide insight into the development of grid-level energy storage systems.

Energy Management Control StraEnergy Management Control Strategy for Renewable Energy System Based on Spotted Hyena Optimizertegy for Renewable Energy System Based on Spotted Hyena Optimizer

Hydrocarbons,carbon monoxide and other pollutants from the transportation sector harm human health in many ways.Fuel cell(FC)has been evolving rapidly over the past two decades due to its efficient mechanism to transform the chemical energy in hydrogen-rich compounds into electrical energy.The main drawback of the standalone FC is its slow dynamic response and its inability to supply rapid variations in the load demand.Therefore,adding energy storage systems is necessary.However,to manage and distribute the power-sharing among the hybrid proton exchange membrane(PEM)fuel cell(FC),battery storage(BS),and supercapacitor(SC),an energy management strategy(EMS)is essential.In this research work,an optimal EMS based on a spotted hyena optimizer(SHO)for hybrid PEM fuel cell/BS/SC is proposed.The main goal of an EMS is to improve the performance of hybrid FC/BS/SC and to reduce the amount of hydrogen consumption.To prove the superiority of the SHO method,the obtained results are compared with the chimp optimizer(CO),the artificial ecosystem-based optimizer(AEO),the seagull optimization algorithm(SOA),the sooty tern optimization algorithm(STOA),and the coyote optimization algorithm(COA).Two main metrics are used as a benchmark for the comparison:the minimum consumed hydrogen and the efficiency of the system.The main findings confirm that the minimum amount of hydrogen consumption and maximum efficiency are achieved by the proposed SHO based EMS.

Role of outdoor trees on pedestrian wind and thermal conditions around a pre-education building for sustainable energy management

Finding sustainable energy resources is essential to face the increasing energy demand.Trees are an important part of ancient architecture but are becoming rare in urban areas.Trees can control and tune the pedestrian-level wind velocity and thermal condition.In this study,a numerical investigation is employed to assess the role of trees planted in the windward direction of the building complex on the thermal and pedestrian wind velocity conditions around/inside a pre-education building located in the center of the complex.Compared to the previous studies(which considered only outside buildings),this work considers the effects of trees on microclimate change both inside/outside buildings.Effects of different parameters including the leaf area density and number of trees,number of rows,far-field velocity magnitude,and thermal condition around the main building are assessed.The results show that the flow velocity in the spacing between the first-row buildings is reduced by 30%-40% when the one-row trees with 2 m height are planted 15 m farther than the buildings.Furthermore,two rows of trees are more effective in higher velocities and reduce the maximum velocity by about 50%.The investigation shows that trees also could reduce the temperature by about 1℃around the building.

Energy Management for a Residential Microgrid Using Wavelet Transform and Fuzzy Control Including a Vehicle-to-Grid System

This paper presents the design and implementation of an energy management system （EMS） with wavelet transform and fuzzy control for a residential micro-grid. The hybrid system in this paper consists of a wind turbine generator, photovoltaic （PV） panels, an electric vehicle （EV）, and a super capacitor （SC）, which is able to connect or disconnect to the main grid. The control strategy is responsible for compensating the difference between the generated power by the wind and solar generators and the demanded power by the loads. Wavelet transform decomposes the power difference into a smoothed component and a fast fluctuated component. The command approach used for fuzzy logic rules considers the state of charging （SOC） of EV, renewable production, and the load demand as parameters. Furthermore, the command rules are developed in order to ensure a reliable grid when taking into account the EV battery protection to decide the output power of the EV. The model of the hybrid system is developed in detail under Matlab/Simulink software environment.

Exploring the impacts of major events on the systemic risk of the international energy market

This study examines the systemic risk caused by major events in the international energy market(IEM)and proposes a management strategy to mitigate it. Using the tail-event driven network(TENET)method, this study constructed a tail-risk spillover network(TRSN) of IEM and simulated the dynamic spillover tail-risk process through the cascading failure mechanism. The study found that renewable energy markets contributed more to systemic risk during the Paris Agreement and the COVID-19pandemic, while fossil energy markets played a larger role during the Russia-Ukraine conflict. This study identifies systemically important markets(SM) and critical tail-risk spillover paths as potential sources of systemic risk. The research confirms that cutting off the IEM risk spillover path can greatly reduce systemic risk and the influence of SM. This study offers insights into the management of systemic risk in IEM and provides policy recommendations to reduce the impact of shock events.

Market Operation of Energy Storage System in Smart Grid:A Review

As a flexible resource,energy storage plays an increasingly significant role in stabilizing and supporting the power system,while providing auxiliary services.Still,the current high demand for energy storage contrasts with the fuzzy lack of market-oriented mechanisms for energy storage,the principle of market-oriented operation has not been embodied,and there is no unified and systematic analytical framework for the business model.However,the dispatch management model of energy storage in actual power system operation is not clear.Still,the specific scheduling process and energy storage strategy on the source-load-network side could be more specific,and there needs to be a greater understanding of the collaborative scheduling process of the multilevel scheduling center.On this basis,this paper reviews the energy storage operation model and market-based incentive mechanism,For different functional types and installation locations of energy storage within the power system,the operational models and existing policies for energy storage participation in the market that are adapted to multiple operating states are summarized.From the point of view of the actual scheduling and operation management of energy storage in China,an energy storage regulation and operation management model based on“national,provincial,and local”multilevel coordination is proposed,as well as key technologies in the interactive scenarios of source-load,network and storage.

An Advanced Energy Management System

A brief outline of Energy Management System (EMS) in China is given and OPEN-2000 EMS, combined with the novel technique of computer science, is introduced. The system has been developed by Power System Control Division of NARI and put into operation in the middle of 1998.

Sustainable Development of Energy Systems and Climate Systems:Key Issues and Perspectives

Climate change and energy security issues are prominent challenges in current energy system management,which should be governed synergistically due to the feedback relationships between them.The“Energy Systems Management and Climate Change”Special Collection Issue in the journal of Energy Engineering provide insights into the field of energy systems management and climate change.From an extended perspective,this study discusses the key issues,research methods and models for energy system management and climate change research.Comprehensive and accurate prediction of energy supply and demand,the evaluation on the energy system resilience to climate change and the coupling methodology application of both nature and social science field maybe the frontier topics around achieving sustainable development goals of energy systems.