Reinforcement Learning Model for Energy System Management to Ensure Energy Efficiency and Comfort in Buildings

This article focuses on the challenges ofmodeling energy supply systems for buildings,encompassing both methods and tools for simulating thermal regimes and engineering systems within buildings.Enhancing the comfort of living or working in buildings often necessitates increased consumption of energy and material,such as for thermal upgrades,which consequently incurs additional economic costs.It is crucial to acknowledge that such improvements do not always lead to a decrease in total pollutant emissions,considering emissions across all stages of production and usage of energy and materials aimed at boosting energy efficiency and comfort in buildings.In addition,it explores the methods and mechanisms for modeling the operating modes of electric boilers used to collectively improve energy efficiency and indoor climatic conditions.Using the developed mathematical models,the study examines the dynamic states of building energy supply systems and provides recommendations for improving their efficiency.These dynamic models are executed in software environments such as MATLAB/Simscape and Python,where the component detailing schemes for various types of controllers are demonstrated.Additionally,controllers based on reinforcement learning(RL)displayed more adaptive load level management.These RL-based controllers can lower instantaneous power usage by up to 35%,reduce absolute deviations from a comfortable temperature nearly by half,and cut down energy consumption by approximately 1%while maintaining comfort.When the energy source produces a constant energy amount,the RL-based heat controllermore effectively maintains the temperature within the set range,preventing overheating.In conclusion,the introduced energydynamic building model and its software implementation offer a versatile tool for researchers,enabling the simulation of various energy supply systems to achieve optimal energy efficiency and indoor climate control in buildings.

Generative pre-trained transformers(GPT)-based automated data mining for building energy management:Advantages,limitations and the future

Advanced data mining methods have shown a promising capacity in building energy management.However,in the past decade,such methods are rarely applied in practice,since they highly rely on users to customize solutions according to the characteristics of target building energy systems.Hence,the major barrier is that the practical applications of such methods remain laborious.It is necessary to enable computers to have the human-like ability to solve data mining tasks.Generative pre-trained transformers(GPT)might be capable of addressing this issue,as some GPT models such as GPT-3.5 and GPT-4 have shown powerful abilities on interaction with humans,code generation,and inference with common sense and domain knowledge.This study explores the potential of the most advanced GPT model(GPT-4)in three data mining scenarios of building energy management,i.e.,energy load prediction,fault diagnosis,and anomaly detection.A performance evaluation framework is proposed to verify the capabilities of GPT-4 on generating energy load prediction codes,diagnosing device faults,and detecting abnormal system operation patterns.It is demonstrated that GPT-4 can automatically solve most of the data mining tasks in this domain,which overcomes the barrier of practical applications of data mining methods in this domain.In the exploration of GPT-4,its advantages and limitations are also discussed comprehensively for revealing future research directions in this domain.

An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

Deep Reinforcement Learning(DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers(RBCs),but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process.Transfer Learning(TL)is a potential solution to address this limitation.However,existing TL applications in building control have been mostly tested among buildings with similar features,not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems.This paper assesses the performance of an online heterogeneous TL strategy,comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python.The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage(TES).The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems,different building envelope features,occupancy schedule and boundary conditions(e.g.,weather and price signal).The TL approach incorporates model slicing,imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings.Results show that the proposed methodology leads to a reduction of 10% in electricity cost and between 10% and 40% in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers.Moreover,online TL maximises self-sufficiency and self-consumption by 9% and 11% with respect to RBC.Conversely,online TL achieves worse performance compared to offline DRL in either transductive or inductive settings.However,offline Deep Reinforcement Learning(DRL)agents should be trained at least for 15 episodes to reach the same level of performance as the online TL.Therefore,the proposed online TL methodology is effective,completely model-free and it can be directly implemented in real buildings with satisfying performance.

Smart Buildings for A Sustainable Development

The use of sustainable technologies for buildings, with the goal of creating an environment for living and working that uses fewer resources and generates less waste, also aims to retrofit existing buildings to be more efficient in terms of energy and water. Many cities are following this way targeting both commercial and municipal buildings. These cities are called smart cities where all life processes and nerve centers of social life are read, in order to radically improve quality of life, opportunity, prosperity, social and economic development, thanks to the use of technology. This paper deals with the study of smart buildings within smart cities, namely the use in an integrated project of computer and telematics tools with automation organized systems and passive bioclimatic strategies in architecture, determining a socio-technical management of intelligent building. The article is the result of a research carded out within the framework of intelligent buildings in the last generation cities, such as those ones with zero emissions that are taking place in the Middle East countries （Dubai, Masdar, Tiajin, and Kochi）. The topic deals with the issues of building automation as a form of technological intelligence and the study of those smart technologies integrated into the building envelope that improve its performances, making it more sustainable. The research methodology has provided a bibliographic retrieval on the state of the art and the latest technological trends in the building field, later has followed a theoretical and comparative approach of the examined technologies, which led to the development of reasoning on operation, performance and functional capabilities of a building that is both sustainable and home automation, to arrive at the final concept of sustainable intelligent building, able to combine the artificial intelligence, home automation, and technological devices of the architectural project to enhance the building energy performance. In conclusion, the proposed result is that of an integrated intelligent building in which artificial intelligence will become part of the shell-building in order to achieve high levels of energy efficiency and thus environmental sustainability.

A PARAMETRIC STUDY OF THE ENERGY EFFICIENCY OF EXISTING AIR-CONDITIONED BUILDINGS IN KUWAIT

In Kuwait air-conditioning systems consume approximately 70%of the grid energy during the long summer months.In this paper,various practical approaches are investigated to enhance the energy efficiency and decrease the energy consumption of five existing air-conditioned(AC)buildings in Kuwait City.The process of energy management in air-conditioned buildings in Kuwait is overwhelming due to high energy consumption in the building sector.This study proposed an optimization technique for the proper energy management of installed AC systems to target energy-efficient buildings.In this study the aim is to explore the effect of different operating parameters,both theoretically and experimentally,and to contribute to the reduction of AC energy consumption.Consequently,the relationship between the thermal load in the air-conditioned buildings and the actual electrical energy consumption is determined,and remedial measures,along with different recom-mendations for energy saving,are presented.The actual thermal loads of each selected building were calculated and compared with the installed cooling capacity of the AC systems.From the results obtained it was concluded that,by implement-ing the suggested remedial measures,the predicted load in the selected buildings could be decreased significantly from the existing installed capacity of the cooling systems.Most of the remedial measures suggested for energy management lead to a reduction in power consumption and increased energy efficiency at different levels based on the specifications of the buildings considered and the AC systems installed,resulting in improved in economy,a reduced carbon footprint,and a cleaner environment.

Distributed Real-time Temperature and Energy Control of Energy Efficient Buildings via Geothermal Heat Pumps

Geothermal heat pumps(GHPs)are a type of heating ventilation and air conditioning(HVAC)systems that use low-temperature resources from soil and groundwater for heating/cooling.In recent years,there has been an increasing interest in GHP systems due to their high energy efficiency and abundant geothermal resources.Thus,the optimization and control design of the GHP system has become a hot topic.On the other hand,as the GHP system is an ideal respon-sive load,mechanism design for the GHP system to realize demand response(DR)in a virtual power plant(VPP)without affecting user comfort is particularly essential.In this paper,we propose a distributed real-time temperature and energy management method via GHP systems for multi-buildings,where both floor and radiator heating/cooling distribution subsystems in multiple thermal zones are considered.We design an energy demand response mechanism for a single GHP to track the given energy consumption command for participating in VPP aggregation/disaggregation.Besides,a coordination mechanism for multiple GHPs is designed for the community-level oper-ator in joining VPP aggregation/disaggregation.Both designed schemes are scalable and do not need to measure or predict any exogenous disturbances such as outdoor temperature and heating disturbances from external sources,e.g.,user activity and device operation.Finally,four numerical examples for the simulation of two different scenarios demonstrate the effectiveness of the proposed methods.

Impact of occupant autonomy on satisfaction and building energy efficiency

The philosophy of building energy management is going through a paradigm change from traditional,often inefficient,user-controlled systems to one that is centrally automated with the aid of IoT-enabled technologies.In this context,occupants’perceived control and building automation may seem to be in conflict.The inquiry of this study is rooted in a proposition that while building automation and centralized control systems are assumed to provide indoor comfort and conserve energy use,limiting occupants’control over their work environment may result in dissatisfaction,and in turn decrease productivity.For assessing this hypothesis,data from the post-occupancy evaluation survey of a smart building in a university in Australia was used to analyze the relationships between perceived control,satisfaction,and perceived productivity.Using structural equation modeling,we have found a positive direct effect of occupants’perceived control on overall satisfaction with their working area.Meanwhile,perceived control exerts an influence on perceived productivity through satisfaction.Furthermore,a field experiment conducted in the same building revealed the potential impact that occupant controllability can have on energy saving.We changed the default light settings from automatic on-and-offto manual-on and automatic-off,letting occupants choose themselves whether to switch the light on or not.Interestingly,about half of the participants usually kept the lights off,preferring daylight in their rooms.This also resulted in a reduction in lighting electricity use by 17.8%without any upfront investment and major technical modification.These findings emphasize the important role of perceived control on occupant satisfaction and productivity,as well as on the energy-saving potential of the user-in-the-loop automation of buildings.

LSTM-based Energy Management for Electric Vehicle Charging in Commercial-building Prosumers

As typical prosumers,commercial buildings equipped with electric vehicle(EV)charging piles and solar photovoltaic panels require an effective energy management method.However,the conventional optimization-model-based building energy management system faces significant challenges regarding prediction and calculation in online execution.To address this issue,a long short-term memory(LSTM)recurrent neural network(RNN)based machine learning algorithm is proposed in this paper to schedule the charging and discharging of numerous EVs in commercial-building prosumers.Under the proposed system control structure,the LSTM algorithm can be separated into offline and online stages.At the offline stage,the LSTM is used to map states(inputs)to decisions(outputs)based on the network training.At the online stage,once the current state is input,the LSTM can quickly generate a solution without any additional prediction.A preliminary data processing rule and an additional output filtering procedure are designed to improve the decision performance of LSTM network.The simulation results demonstrate that the LSTM algorithm can generate near-optimal solutions in milliseconds and significantly reduce the prediction and calculation pressures compared with the conventional optimization algorithm.

A framework for building energy management system with residence mounted photovoltaic

Efficient utilization of a residential photovoltaic (PV) array with grid connection is difficult due to power fluctuation and geographical dispersion. Reliable energy management and control system are required for overcoming these obstacles. This study provides a new residential energy management system (REMS) based on the convolution neural network (CNN) including PV array environment. The CNN is used in the estimation of the nonlinear relationship between the residence PV array power and meteorological datasets. REMS has three main stages for the energy management such as forecasting, scheduling, and real functioning. A short term forecasting strategy has been performed in the forecasting stage based on the PV power and the residential load. A coordinated scheduling has been utilized for minimizing the functioning cost. A real-time predictive strategy has been used in the actual functioning stage to minimize the difference between the actual and scheduled power consumption of the building. The proposed approach has been evaluated based on real-time power and meteorological data sets.

Machine Learning and Deep Learning Methods for Enhancing Building Energy Efficiency and Indoor Environmental Quality – A Review

The built environment sector is responsible for almost one-third of the world’s final energy consumption. Hence, seeking plausible solutions to minimise building energy demands and mitigate adverse environmental impacts is necessary. Artificial intelligence (AI) techniques such as machine and deep learning have been increasingly and successfully applied to develop solutions for the built environment. This review provided a critical summary of the existing literature on the machine and deep learning methods for the built environment over the past decade, with special reference to holistic approaches. Different AI-based techniques employed to resolve interconnected problems related to heating, ventilation and air conditioning (HVAC) systems and enhance building performances were reviewed, including energy forecasting and management, indoor air quality and occupancy comfort/satisfaction prediction, occupancy detection and recognition, and fault detection and diagnosis. The present study explored existing AI-based techniques focusing on the framework, methodology, and performance. The literature highlighted that selecting the most suitable machine learning and deep learning model for solving a problem could be challenging. The recent explosive growth experienced by the research area has led to hundreds of machine learning algorithms being applied to building performance-related studies. The literature showed that existing research studies considered a wide range of scope/scales (from an HVAC component to urban areas) and time scales (minute to year). This makes it difficult to find an optimal algorithm for a specific task or case. The studies also employed a wide range of evaluation metrics, adding to the challenge. Further developments and more specific guidelines are required for the built environment field to encourage best practices in evaluating and selecting models. The literature also showed that while machine and deep learning had been successfully applied in building energy efficiency research, most of the studies are still at the experimental or testing stage, and there are limited studies which implemented machine and deep learning strategies in actual buildings and conducted the post-occupancy evaluation.

Machine learning in building energy management: A critical review and future directions

Over the past two decades,machine learning(ML)has elicited increasing attention in building energy management(BEM)research.However,the boundary of the ML-BEM research has not been clearly defined,and no thorough review of ML applications in BEM during the whole building life-cycle has been published.This study aims to address this gap by reviewing the ML-BEM papers to ascertain the status of this research area and identify future research directions.An integrated framework of ML-BEM,composed of four layers and a series of driving factors,is proposed.Then,based on the hype cycle model,this paper analyzes the current development status of ML-BEM and tries to predict its future development trend.Finally,five research directions are discussed:(1)the behavioral impact on BEM,(2)the integration management of renewable energy,(3)security concerns of ML-BEM,(4)extension to other building life-cycle phases,and(5)the focus on fault detection and diagnosis.The findings of this study are believed to provide useful references for future research on ML-BEM.

Control of Indoor Swimming Pools with Potential for Demand Response

Buildings with indoor swimming pools are recognized as very high-energy consumers and present a great potential for electrical and thermal energy savings. A BEMS （building energy management system） could be conceived in order to optimize the building energy demand and with smart grid interaction. This paper presents the condition and potential contract-based demand side response in indoor swimming pools context. The BEMS designed by the authors implements control strategies for HVAC （heating, ventilation and air conditioning） and pumping system in order to reduce the electricity demand during peak hours or in response to an emergency signal from the system operator in critical times. The control strategies for HVAC was carried out by Building Thermal Simulation and the used of a theoretical formula for pumping system, strategies can carry out a significant reduction in power demand both in HVAC and pumping systems.

Novel machine learning paradigms-enabled methods for smart building operations in data-challenging contexts:Progress and perspectives

The increasing availability of building operational data has greatly encouraged the development of advanced data-driven technologies for smart building operations.Building operational data typically suffer from data quality problems,such as insufficient labeled and imbalanced data,making them incompatible with conventional machine learning algorithms.Recent advances in data science have provided novel machine learning paradigms to tackle such data challenges for practical applications,such as transfer learning,semi-supervised learning,and generative learning.This review aims to present the progress and perspectives on the effective utilization of novel machine learning paradigms for three major building energy management tasks,i.e.,building energy predictions,fault detection and diagnosis,and control optimizations.In-depth discussions have been provided to demonstrate the pros and cons of different learning approaches in terms of data compatibility,modeling difficulties,and possible application scenarios,which could be helpful for enhancing the feasibility of data-driven technologies for smart building operations.

Internal Combustion Engine as a New Source for Enhancing Distribution System Resilience

Enhancing distribution system resilience is a new challenge for researchers.Supplying distribution loads,especially the residential customers and high-priority loads after disasters,is vital for this purpose.In this paper,the internal combustion engine(ICE)vehicles are firstly introduced as valuable backup energy sources in the aftermath of disasters and the use of this technology is explained.Then,the improvement of distribution system resilience is investigated through supplying smart residential customers and injecting extra power to the main grid.In this method,it is assumed that the infrastructure of distribution system is partially damaged(common cases)and it can be restored in less than one day.The extra power of residential customer can be delivered to other loads.A novel formulation for increasing the injected power of the smart home to the main grid using ICE vehicles is proposed.Moreover,the maximum backup duration in case of extensive damages in the distribution system is calculated for some commercial ICE vehicles.In this case,the smart home cannot deliver extra energy to the main grid because of its survivability.Simulation results demonstrate the effectiveness of the proposed method for increasing backup power during power outages.It is also shown that ICE vehicles can supply residential customers for a reasonable amount of time during a power outage.