Reinforcement Learning Model for Energy System Management to Ensure Energy Efficiency and Comfort in Buildings

This article focuses on the challenges ofmodeling energy supply systems for buildings,encompassing both methods and tools for simulating thermal regimes and engineering systems within buildings.Enhancing the comfort of living or working in buildings often necessitates increased consumption of energy and material,such as for thermal upgrades,which consequently incurs additional economic costs.It is crucial to acknowledge that such improvements do not always lead to a decrease in total pollutant emissions,considering emissions across all stages of production and usage of energy and materials aimed at boosting energy efficiency and comfort in buildings.In addition,it explores the methods and mechanisms for modeling the operating modes of electric boilers used to collectively improve energy efficiency and indoor climatic conditions.Using the developed mathematical models,the study examines the dynamic states of building energy supply systems and provides recommendations for improving their efficiency.These dynamic models are executed in software environments such as MATLAB/Simscape and Python,where the component detailing schemes for various types of controllers are demonstrated.Additionally,controllers based on reinforcement learning(RL)displayed more adaptive load level management.These RL-based controllers can lower instantaneous power usage by up to 35%,reduce absolute deviations from a comfortable temperature nearly by half,and cut down energy consumption by approximately 1%while maintaining comfort.When the energy source produces a constant energy amount,the RL-based heat controllermore effectively maintains the temperature within the set range,preventing overheating.In conclusion,the introduced energydynamic building model and its software implementation offer a versatile tool for researchers,enabling the simulation of various energy supply systems to achieve optimal energy efficiency and indoor climate control in buildings.

Automated recognition and mapping of building management system (BMS) data points for building energy modeling (BEM)

With the advance of the internet of things and building management system(BMS)in modern buildings,there is an opportunity of using the data to extend the use of building energy modeling(BEM)beyond the design phase.Potential applications include retrofit analysis,measurement and verification,and operations and controls.However,while BMS is collecting a vast amount of operation data,different suppliers and sensor installers typically apply their own customized or even random non-uniform rules to define the metadata,i.e.,the point tags.This results in a need to interpret and manually map any BMS data before using it for energy analysis.The mapping process is labor-intensive,error-prone,and requires comprehensive prior knowledge.Additionally,BMS metadata typically has considerable variety and limited context information,limiting the applicability of existing interpreting methods.In this paper,we proposed a text mining framework to facilitate interpreting and mapping BMS points to EnergyPlus variables.The framework is based on unsupervised density-based clustering(DBSCAN)and a novel fuzzy string matching algorithm“X-gram”.Therefore,it is generalizable among different buildings and naming conventions.We compare the proposed framework against commonly used baselines that include morphological analysis and widely used text mining techniques.Using two building cases from Singapore and two from the United States,we demonstrated that the framework outperformed baseline methods by 25.5%,with the measurement extraction F-measure of 87.2%and an average mapping accuracy of 91.4%.

Fuzzy-AHP-Based Comprehensive Evaluation on Facility Management System of High-Rise Office Buildings

The present building facility management status in China resulted in many problems such as high energy consumption,failure of automation control,services failure and poor indoor air quality.Based on questionnaires and interviews to professional engineers and building users,a comprehensive evaluation index system was established on facility management of high-rise office buildings.A Fuzzy AHP based upon hierarchy criteria system was established.A Fuzzy AHP Evaluation Model on Facility Management System was set up;α-cut analysis was introduced and incorporated with expert knowledge together,which made up the optimism index λ.The fuzzy optimum crisp weight of each criterion was resulted from data-mining.Case investigations were processed in high-rise office buildings in Shenyang.The results illustrated that indoor air quality,thermal comfort and life cycle cost were the most important indexes in the evaluation of Facility Management System of high rise office buildings.Residents in high-rise buildings in Shenyang pay less attention to maintenance management and environment protection.By comparison with the analysis result of Export Choice,Fuzzy AHP-based evaluation model could act as a scientific reference for the establishment of governmental standards in facility management area in building.

Virtual Reality Used in the Lighting System Management of a Building

The first component of a building implemented in a virtual prototype concerning the management of a building is a lighting system. It was applied in a study case. The interactive application allows the examination of the physical model, visualizing, for each element modeled in three-dimensions （3D） and linked to a database, the corresponding technical information concerned with the use of the material, calculated for different points in time during their life. The control of a lamp stock, the constant updating of lifetime information and the planning of periodical local inspections are attended on the prototype. This is an important mean of cooperation between collaborators involved in the building management.

Design and Management of Building＇s Resources

The developed modem control systems and buildings management resource systems would be effective if they are based on previously established optimal conditions during the building design. This is one of the key issues for a responsible architecture. The focus of this paper is on sustainable design methods and techniques for saving resources and their management throughout the building lifecycle. The main subject of the present article is the characteristics of these methods and their fundamental role in sustainable resource management during the building operation. The results which are based on conducted case studies of European and international practice in the construction of sustainable buildings are implemented here. Key features of a comprehensive approach for design and construction are outlined via comparative analysis, as well as various systems for the evaluation of sustainability for already constructed buildings. The mostly used criteria and indicators for sustainability are systematized, including those related to resource consumption. By analyzing a specific example, the role of sustainable design methods is justified as an important prerequisite for effective management of building resources in the process building maintenance. According to the conducted studies, during the longest life cycle period of a building, by implementation of control systems and resource management of building, the costs are successfully optimized. Specific directions that prove the effectiveness of such systems are systematized in the paper. Innovative approaches, complex methods and measures for design and management of buildings resources are presented as results of this study.

Generative pre-trained transformers(GPT)-based automated data mining for building energy management:Advantages,limitations and the future

Advanced data mining methods have shown a promising capacity in building energy management.However,in the past decade,such methods are rarely applied in practice,since they highly rely on users to customize solutions according to the characteristics of target building energy systems.Hence,the major barrier is that the practical applications of such methods remain laborious.It is necessary to enable computers to have the human-like ability to solve data mining tasks.Generative pre-trained transformers(GPT)might be capable of addressing this issue,as some GPT models such as GPT-3.5 and GPT-4 have shown powerful abilities on interaction with humans,code generation,and inference with common sense and domain knowledge.This study explores the potential of the most advanced GPT model(GPT-4)in three data mining scenarios of building energy management,i.e.,energy load prediction,fault diagnosis,and anomaly detection.A performance evaluation framework is proposed to verify the capabilities of GPT-4 on generating energy load prediction codes,diagnosing device faults,and detecting abnormal system operation patterns.It is demonstrated that GPT-4 can automatically solve most of the data mining tasks in this domain,which overcomes the barrier of practical applications of data mining methods in this domain.In the exploration of GPT-4,its advantages and limitations are also discussed comprehensively for revealing future research directions in this domain.

An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

Deep Reinforcement Learning(DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers(RBCs),but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process.Transfer Learning(TL)is a potential solution to address this limitation.However,existing TL applications in building control have been mostly tested among buildings with similar features,not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems.This paper assesses the performance of an online heterogeneous TL strategy,comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python.The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage(TES).The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems,different building envelope features,occupancy schedule and boundary conditions(e.g.,weather and price signal).The TL approach incorporates model slicing,imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings.Results show that the proposed methodology leads to a reduction of 10% in electricity cost and between 10% and 40% in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers.Moreover,online TL maximises self-sufficiency and self-consumption by 9% and 11% with respect to RBC.Conversely,online TL achieves worse performance compared to offline DRL in either transductive or inductive settings.However,offline Deep Reinforcement Learning(DRL)agents should be trained at least for 15 episodes to reach the same level of performance as the online TL.Therefore,the proposed online TL methodology is effective,completely model-free and it can be directly implemented in real buildings with satisfying performance.

Optimization Methodology of ESGB on Weights,Evaluation Rating,and Calculation of Carbon Emissions

To cope with climate change,the Chinese government took the lead in advocating the aim of a Carbon peak by 2030 and carbon neutrality by 2060.In China,the total carbon emission of the whole construction process in 2020 was about 5.08 billion tons of carbon dioxide(CO_(2)),accounting for about 50.90%of the national carbon emission.Consequently,researchers come up with a series of standard assessments for green building optimization measures.Through analysis and comparison of Leadership in Energy and Environmental Design(LEED),WELL Building Standard(WELL),Building Research Establishment Environmental Assessment Method(BREEAM),and Evaluation Standard for Green Building(ESGB)standards,this study will draw conclusions on optimizing ESGB in terms of weighting,evaluation rating,and carbon emission calculations.

Optimization of Grouping Evacuation Strategy in High-rise Building Fires Based on Graph Theory and Computational Experiments

It is difficult to rescue people from outside, and emergency evacuation is still a main measure to decrease casualties in high-rise building fires. To improve evacuation efficiency, a valid and easily manipulated grouping evacuation strategy is proposed. Occupants escape in groups according to the shortest evacuation route is determined by graph theory. In order to evaluate and find the optimal grouping, computational experiments are performed to design and simulate the evacuation processes. A case study shown the application in detail and quantitative research conclusions is obtained. The thoughts and approaches of this study can be used to guide actual high-rise building evacuation processes in future.

Smart Buildings for A Sustainable Development

The use of sustainable technologies for buildings, with the goal of creating an environment for living and working that uses fewer resources and generates less waste, also aims to retrofit existing buildings to be more efficient in terms of energy and water. Many cities are following this way targeting both commercial and municipal buildings. These cities are called smart cities where all life processes and nerve centers of social life are read, in order to radically improve quality of life, opportunity, prosperity, social and economic development, thanks to the use of technology. This paper deals with the study of smart buildings within smart cities, namely the use in an integrated project of computer and telematics tools with automation organized systems and passive bioclimatic strategies in architecture, determining a socio-technical management of intelligent building. The article is the result of a research carded out within the framework of intelligent buildings in the last generation cities, such as those ones with zero emissions that are taking place in the Middle East countries （Dubai, Masdar, Tiajin, and Kochi）. The topic deals with the issues of building automation as a form of technological intelligence and the study of those smart technologies integrated into the building envelope that improve its performances, making it more sustainable. The research methodology has provided a bibliographic retrieval on the state of the art and the latest technological trends in the building field, later has followed a theoretical and comparative approach of the examined technologies, which led to the development of reasoning on operation, performance and functional capabilities of a building that is both sustainable and home automation, to arrive at the final concept of sustainable intelligent building, able to combine the artificial intelligence, home automation, and technological devices of the architectural project to enhance the building energy performance. In conclusion, the proposed result is that of an integrated intelligent building in which artificial intelligence will become part of the shell-building in order to achieve high levels of energy efficiency and thus environmental sustainability.

Energy management of buildings with energy storage and solar photovoltaic:A diversity in experience approach for deep reinforcement learning agents

Deep reinforcement learning(DRL)is a suitable approach to handle uncertainty in managing the energy consumption of buildings with energy storage systems.Conventionally,DRL agents are trained by randomly selecting samples from a data set,which can result in overexposure to some data categories and under/no exposure to other data categories.Thus,the trained model may be biased towards some data groups and underperform(provide suboptimal results)for data groups to which it was less exposed.To address this issue,diversity in experience-based DRL agent training framework is proposed in this study.This approach ensures the exposure of agents to all types of data.The proposed framework is implemented in two steps.In the first step,raw data are grouped into different clusters using the K-means clustering method.The clustered data is then arranged by stacking the data of one cluster on top of another.In the second step,a selection algorithm is proposed to select data from each cluster to train the DRL agent.The frequency of selection from each cluster is in proportion to the number of data points in that cluster and therefore named the proportional selection method.To analyze the performance of the proposed approach and compare the results with the conventional random selection method,two indices are proposed in this study:the flatness index and the divergence index.The model is trained using different data sets(1-year,3-year,and 5-year)and also with the inclusion of solar photovoltaics.The simulation results confirmed the superior performance of the proposed approach to flatten the building’s load curve by optimally operating the energy storage system.

How to Build the Management Mode for the Gymnasiums in Ordinary Universities in China

In recent years, China has paid attention to the college expansion plan, and people have attached importance not to only students＇ academic performance, but also their quality and extracurricular activities. At this point, all kinds of activities for college students have been widely concerned. Students in ordinary universities almost exercise in the gymnasium, which requires optimizing the management mode of gymnasium construction. In the process of university gymnasium management, it needs building effectively according to the requirements of college students＇ activities, ensuring a better activity space for college students. At this point, it is necessary for relevant management personnel to focus on the management, and make perfect management mode, so as to ensure the better optimal management system. This paper proposes measures to build the university gymnasium management mode through the analysis of the university gymnasium management mode problems, in order to offer reference for management personnel.

Consumer Central Energy Flexibility in Office Buildings

Energy flexibility in buildings will play an important role in the smart energy system. Office buildings have more potentials to provide energy flexibility to the grid compared to other types of buildings, due to the existing building management, control systems and large energy consumption. Consumers in office buildings （building owners/managers and occupants） take a main role for adopting and engaging in building energy flexibility. This paper provides a systematic review of consumer central energy flexibility in office buildings with the discussion of social, technical and business aspects. This paper clarifies the correlations of consumers＇ concems, external influential factors, energy flexibility resources and technology with eight hypotheses. This paper suggests that technical solutions with the integration of distributed energy resources, building management and control system can boost energy flexibility in the office buildings.

A framework for building energy management system with residence mounted photovoltaic

Efficient utilization of a residential photovoltaic (PV) array with grid connection is difficult due to power fluctuation and geographical dispersion. Reliable energy management and control system are required for overcoming these obstacles. This study provides a new residential energy management system (REMS) based on the convolution neural network (CNN) including PV array environment. The CNN is used in the estimation of the nonlinear relationship between the residence PV array power and meteorological datasets. REMS has three main stages for the energy management such as forecasting, scheduling, and real functioning. A short term forecasting strategy has been performed in the forecasting stage based on the PV power and the residential load. A coordinated scheduling has been utilized for minimizing the functioning cost. A real-time predictive strategy has been used in the actual functioning stage to minimize the difference between the actual and scheduled power consumption of the building. The proposed approach has been evaluated based on real-time power and meteorological data sets.

Novel machine learning paradigms-enabled methods for smart building operations in data-challenging contexts:Progress and perspectives

The increasing availability of building operational data has greatly encouraged the development of advanced data-driven technologies for smart building operations.Building operational data typically suffer from data quality problems,such as insufficient labeled and imbalanced data,making them incompatible with conventional machine learning algorithms.Recent advances in data science have provided novel machine learning paradigms to tackle such data challenges for practical applications,such as transfer learning,semi-supervised learning,and generative learning.This review aims to present the progress and perspectives on the effective utilization of novel machine learning paradigms for three major building energy management tasks,i.e.,building energy predictions,fault detection and diagnosis,and control optimizations.In-depth discussions have been provided to demonstrate the pros and cons of different learning approaches in terms of data compatibility,modeling difficulties,and possible application scenarios,which could be helpful for enhancing the feasibility of data-driven technologies for smart building operations.

A PARAMETRIC STUDY OF THE ENERGY EFFICIENCY OF EXISTING AIR-CONDITIONED BUILDINGS IN KUWAIT

In Kuwait air-conditioning systems consume approximately 70%of the grid energy during the long summer months.In this paper,various practical approaches are investigated to enhance the energy efficiency and decrease the energy consumption of five existing air-conditioned(AC)buildings in Kuwait City.The process of energy management in air-conditioned buildings in Kuwait is overwhelming due to high energy consumption in the building sector.This study proposed an optimization technique for the proper energy management of installed AC systems to target energy-efficient buildings.In this study the aim is to explore the effect of different operating parameters,both theoretically and experimentally,and to contribute to the reduction of AC energy consumption.Consequently,the relationship between the thermal load in the air-conditioned buildings and the actual electrical energy consumption is determined,and remedial measures,along with different recom-mendations for energy saving,are presented.The actual thermal loads of each selected building were calculated and compared with the installed cooling capacity of the AC systems.From the results obtained it was concluded that,by implement-ing the suggested remedial measures,the predicted load in the selected buildings could be decreased significantly from the existing installed capacity of the cooling systems.Most of the remedial measures suggested for energy management lead to a reduction in power consumption and increased energy efficiency at different levels based on the specifications of the buildings considered and the AC systems installed,resulting in improved in economy,a reduced carbon footprint,and a cleaner environment.

LSTM-based Energy Management for Electric Vehicle Charging in Commercial-building Prosumers

As typical prosumers,commercial buildings equipped with electric vehicle(EV)charging piles and solar photovoltaic panels require an effective energy management method.However,the conventional optimization-model-based building energy management system faces significant challenges regarding prediction and calculation in online execution.To address this issue,a long short-term memory(LSTM)recurrent neural network(RNN)based machine learning algorithm is proposed in this paper to schedule the charging and discharging of numerous EVs in commercial-building prosumers.Under the proposed system control structure,the LSTM algorithm can be separated into offline and online stages.At the offline stage,the LSTM is used to map states(inputs)to decisions(outputs)based on the network training.At the online stage,once the current state is input,the LSTM can quickly generate a solution without any additional prediction.A preliminary data processing rule and an additional output filtering procedure are designed to improve the decision performance of LSTM network.The simulation results demonstrate that the LSTM algorithm can generate near-optimal solutions in milliseconds and significantly reduce the prediction and calculation pressures compared with the conventional optimization algorithm.

Digital Vision Based Concrete Compressive Strength Evaluating Model Using Deep Convolutional Neural Network

Compressive strength of concrete is a significant factor to assess building structure health and safety.Therefore,various methods have been developed to evaluate the compressive strength of concrete structures.However,previous methods have several challenges in costly,time-consuming,and unsafety.To address these drawbacks,this paper proposed a digital vision based concrete compressive strength evaluating model using deep convolutional neural network(DCNN).The proposed model presented an alternative approach to evaluating the concrete strength and contributed to improving efficiency and accuracy.The model was developed with 4,000 digital images and 61,996 images extracted from video recordings collected from concrete samples.The experimental results indicated a root mean square error(RMSE)value of 3.56(MPa),demonstrating a strong feasibility that the proposed model can be utilized to predict the concrete strength with digital images of their surfaces and advantages to overcome the previous limitations.This experiment contributed to provide the basis that could be extended to future research with image analysis technique and artificial neural network in the diagnosis of concrete building structures.

Machine learning in building energy management: A critical review and future directions

Over the past two decades,machine learning(ML)has elicited increasing attention in building energy management(BEM)research.However,the boundary of the ML-BEM research has not been clearly defined,and no thorough review of ML applications in BEM during the whole building life-cycle has been published.This study aims to address this gap by reviewing the ML-BEM papers to ascertain the status of this research area and identify future research directions.An integrated framework of ML-BEM,composed of four layers and a series of driving factors,is proposed.Then,based on the hype cycle model,this paper analyzes the current development status of ML-BEM and tries to predict its future development trend.Finally,five research directions are discussed:(1)the behavioral impact on BEM,(2)the integration management of renewable energy,(3)security concerns of ML-BEM,(4)extension to other building life-cycle phases,and(5)the focus on fault detection and diagnosis.The findings of this study are believed to provide useful references for future research on ML-BEM.

Distributed Real-time Temperature and Energy Control of Energy Efficient Buildings via Geothermal Heat Pumps

Geothermal heat pumps(GHPs)are a type of heating ventilation and air conditioning(HVAC)systems that use low-temperature resources from soil and groundwater for heating/cooling.In recent years,there has been an increasing interest in GHP systems due to their high energy efficiency and abundant geothermal resources.Thus,the optimization and control design of the GHP system has become a hot topic.On the other hand,as the GHP system is an ideal respon-sive load,mechanism design for the GHP system to realize demand response(DR)in a virtual power plant(VPP)without affecting user comfort is particularly essential.In this paper,we propose a distributed real-time temperature and energy management method via GHP systems for multi-buildings,where both floor and radiator heating/cooling distribution subsystems in multiple thermal zones are considered.We design an energy demand response mechanism for a single GHP to track the given energy consumption command for participating in VPP aggregation/disaggregation.Besides,a coordination mechanism for multiple GHPs is designed for the community-level oper-ator in joining VPP aggregation/disaggregation.Both designed schemes are scalable and do not need to measure or predict any exogenous disturbances such as outdoor temperature and heating disturbances from external sources,e.g.,user activity and device operation.Finally,four numerical examples for the simulation of two different scenarios demonstrate the effectiveness of the proposed methods.