We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zo...We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.展开更多
This paper proposes a novel reinforcement learning(RL)architecture for the efficient scheduling and control of the heating,ventilation and air conditioning(HVAC)system in a commercial building while harnessing its de-...This paper proposes a novel reinforcement learning(RL)architecture for the efficient scheduling and control of the heating,ventilation and air conditioning(HVAC)system in a commercial building while harnessing its de-mand response(DR)potentials.With advances in automated building management systems,this can be achieved seamlessly by a smart autonomous RL agent which takes the best action,for example,a change in HVAC temper-ature set point,necessary to change the electricity usage pattern of a building in response to demand response signals,and with minimal thermal comfort impact to customers.Previous research in this area has tackled only individual aspects of the problem using RL.Specifically,due to the challenges in implementing demand response with whole-building models,simpler analytical models which poorly capture reality have been used instead.And where whole-building models are applied,RL is used for HVAC control mainly to achieve energy efficiency goals while demand response is neglected.Thus,in this research,we implement a holistic framework by designing an efficient RL controller for a whole-building model which learns to optimise and control the HVAC system for improved energy efficiency and thermal comfort levels in addition to achieving demand response goals.Our simulation results show that by applying reinforcement learning for normal HVAC operation,a maximum weekly energy reduction of up to 22%can be achieved compared to a handcrafted baseline controller.Furthermore,by employing a DR-aware RL controller during demand response periods,average power reductions or increases of up to 50%can be achieved on a weekly basis compared to the default RL controller,while keeping occupant thermal comfort levels within acceptable bounds.展开更多
In heating, ventilating and air-conditioning （HVAC） systems, there exist severe nonlinearity, time-varying nature, disturbances and uncertainties. A new predictive functional control based on Takagi-Sugeno （T-S） f...In heating, ventilating and air-conditioning （HVAC） systems, there exist severe nonlinearity, time-varying nature, disturbances and uncertainties. A new predictive functional control based on Takagi-Sugeno （T-S） fuzzy model was proposed to control HVAC systems. The T-S fuzzy model of stabilized controlled process was obtained using the least squares method, then on the basis of global linear predictive model from T-S fuzzy model, the process was controlled by the predictive functional controller. Especially the feedback regulation part was developed to compensate uncertainties of fuzzy predictive model. Finally simulation test results in HVAC systems control applications showed that the proposed fuzzy model predictive functional control improves tracking effect and robustness. Compared with the conventional PID controller, this control strategy has the advantages of less overshoot and shorter setting time, etc.展开更多
Heating, ventilation, and air conditioning (HVAC) system is significant to the energy efficiency in buildings. In this paper, temperature control of HVAC system is studied in winter operation season. The physical mode...Heating, ventilation, and air conditioning (HVAC) system is significant to the energy efficiency in buildings. In this paper, temperature control of HVAC system is studied in winter operation season. The physical model of the zone, the fan, the heating coil and sensor are built. HVAC is a non-linear, strong disturbance and coupling system. Linear active-rejection-disturbance-control is an appreciate control algorithm which can adapt to less information, strong-disturbance influence, and has relative-fixed structure and simple tuning process of the controller parameters. Active-rejection-disturbance-control of the HVAC system is proposed. Simulation in Matlab/Simulink was done. Simulation results show that linear active-rejection-disturbance-control was prior to PID and integral-fuzzy controllers in rising time, overshoot and response time of step disturbance. The study can provide fundamental basis for the control of the air-condition system with strong-disturbance and high-precision needed.展开更多
针对暖通空调(HVAC)系统难以控制的问题,提出一种基于m ax-product推理的M am dan i模糊模型预测控制策略.首先利用一步模糊预测模型的结构分析得到其解析表达式,获得系统在k+1时刻的线性化预测模型;然后基于模糊线性化模型进行模型预...针对暖通空调(HVAC)系统难以控制的问题,提出一种基于m ax-product推理的M am dan i模糊模型预测控制策略.首先利用一步模糊预测模型的结构分析得到其解析表达式,获得系统在k+1时刻的线性化预测模型;然后基于模糊线性化模型进行模型预测控制器设计.对HVAC系统的仿真和实验结果表明,该算法是一种跟踪性能好且鲁棒性强的有效控制算法.展开更多
文摘We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.
基金This research was funded by Australian Renewable Energy Agency(ARENA)as part of ARENA’s Advancing Renewables Program under Grant 2018/ARP017.
文摘This paper proposes a novel reinforcement learning(RL)architecture for the efficient scheduling and control of the heating,ventilation and air conditioning(HVAC)system in a commercial building while harnessing its de-mand response(DR)potentials.With advances in automated building management systems,this can be achieved seamlessly by a smart autonomous RL agent which takes the best action,for example,a change in HVAC temper-ature set point,necessary to change the electricity usage pattern of a building in response to demand response signals,and with minimal thermal comfort impact to customers.Previous research in this area has tackled only individual aspects of the problem using RL.Specifically,due to the challenges in implementing demand response with whole-building models,simpler analytical models which poorly capture reality have been used instead.And where whole-building models are applied,RL is used for HVAC control mainly to achieve energy efficiency goals while demand response is neglected.Thus,in this research,we implement a holistic framework by designing an efficient RL controller for a whole-building model which learns to optimise and control the HVAC system for improved energy efficiency and thermal comfort levels in addition to achieving demand response goals.Our simulation results show that by applying reinforcement learning for normal HVAC operation,a maximum weekly energy reduction of up to 22%can be achieved compared to a handcrafted baseline controller.Furthermore,by employing a DR-aware RL controller during demand response periods,average power reductions or increases of up to 50%can be achieved on a weekly basis compared to the default RL controller,while keeping occupant thermal comfort levels within acceptable bounds.
基金This work was supported by Young Scientists Fundamental Research Program of Shandong Province of China (No. 031B5147).
文摘In heating, ventilating and air-conditioning （HVAC） systems, there exist severe nonlinearity, time-varying nature, disturbances and uncertainties. A new predictive functional control based on Takagi-Sugeno （T-S） fuzzy model was proposed to control HVAC systems. The T-S fuzzy model of stabilized controlled process was obtained using the least squares method, then on the basis of global linear predictive model from T-S fuzzy model, the process was controlled by the predictive functional controller. Especially the feedback regulation part was developed to compensate uncertainties of fuzzy predictive model. Finally simulation test results in HVAC systems control applications showed that the proposed fuzzy model predictive functional control improves tracking effect and robustness. Compared with the conventional PID controller, this control strategy has the advantages of less overshoot and shorter setting time, etc.
文摘Heating, ventilation, and air conditioning (HVAC) system is significant to the energy efficiency in buildings. In this paper, temperature control of HVAC system is studied in winter operation season. The physical model of the zone, the fan, the heating coil and sensor are built. HVAC is a non-linear, strong disturbance and coupling system. Linear active-rejection-disturbance-control is an appreciate control algorithm which can adapt to less information, strong-disturbance influence, and has relative-fixed structure and simple tuning process of the controller parameters. Active-rejection-disturbance-control of the HVAC system is proposed. Simulation in Matlab/Simulink was done. Simulation results show that linear active-rejection-disturbance-control was prior to PID and integral-fuzzy controllers in rising time, overshoot and response time of step disturbance. The study can provide fundamental basis for the control of the air-condition system with strong-disturbance and high-precision needed.
文摘针对暖通空调(HVAC)系统难以控制的问题,提出一种基于m ax-product推理的M am dan i模糊模型预测控制策略.首先利用一步模糊预测模型的结构分析得到其解析表达式,获得系统在k+1时刻的线性化预测模型;然后基于模糊线性化模型进行模型预测控制器设计.对HVAC系统的仿真和实验结果表明,该算法是一种跟踪性能好且鲁棒性强的有效控制算法.
