Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Smart Energy Management System Using Machine Learning

Energy management is an inspiring domain in developing of renewable energy sources.However,the growth of decentralized energy production is revealing an increased complexity for power grid managers,inferring more quality and reliability to regulate electricity flows and less imbalance between electricity production and demand.The major objective of an energy management system is to achieve optimum energy procurement and utilization throughout the organization,minimize energy costs without affecting production,and minimize environmental effects.Modern energy management is an essential and complex subject because of the excessive consumption in residential buildings,which necessitates energy optimization and increased user comfort.To address the issue of energy management,many researchers have developed various frameworks;while the objective of each framework was to sustain a balance between user comfort and energy consumption,this problem hasn’t been fully solved because of how difficult it is to solve it.An inclusive and Intelligent Energy Management System(IEMS)aims to provide overall energy efficiency regarding increased power generation,increase flexibility,increase renewable generation systems,improve energy consumption,reduce carbon dioxide emissions,improve stability,and reduce energy costs.Machine Learning(ML)is an emerging approach that may be beneficial to predict energy efficiency in a better way with the assistance of the Internet of Energy(IoE)network.The IoE network is playing a vital role in the energy sector for collecting effective data and usage,resulting in smart resource management.In this research work,an IEMS is proposed for Smart Cities(SC)using the ML technique to better resolve the energy management problem.The proposed system minimized the energy consumption with its intelligent nature and provided better outcomes than the previous approaches in terms of 92.11% accuracy,and 7.89% miss-rate.

Reinforcement Learning Model for Energy System Management to Ensure Energy Efficiency and Comfort in Buildings

This article focuses on the challenges ofmodeling energy supply systems for buildings,encompassing both methods and tools for simulating thermal regimes and engineering systems within buildings.Enhancing the comfort of living or working in buildings often necessitates increased consumption of energy and material,such as for thermal upgrades,which consequently incurs additional economic costs.It is crucial to acknowledge that such improvements do not always lead to a decrease in total pollutant emissions,considering emissions across all stages of production and usage of energy and materials aimed at boosting energy efficiency and comfort in buildings.In addition,it explores the methods and mechanisms for modeling the operating modes of electric boilers used to collectively improve energy efficiency and indoor climatic conditions.Using the developed mathematical models,the study examines the dynamic states of building energy supply systems and provides recommendations for improving their efficiency.These dynamic models are executed in software environments such as MATLAB/Simscape and Python,where the component detailing schemes for various types of controllers are demonstrated.Additionally,controllers based on reinforcement learning(RL)displayed more adaptive load level management.These RL-based controllers can lower instantaneous power usage by up to 35%,reduce absolute deviations from a comfortable temperature nearly by half,and cut down energy consumption by approximately 1%while maintaining comfort.When the energy source produces a constant energy amount,the RL-based heat controllermore effectively maintains the temperature within the set range,preventing overheating.In conclusion,the introduced energydynamic building model and its software implementation offer a versatile tool for researchers,enabling the simulation of various energy supply systems to achieve optimal energy efficiency and indoor climate control in buildings.

AI-Driven Learning Management Systems:Modern Developments, Challenges and Future Trends during theAge of ChatGPT

COVID-19 pandemic restrictions limited all social activities to curtail the spread of the virus.The foremost and most prime sector among those affected were schools,colleges,and universities.The education system of entire nations had shifted to online education during this time.Many shortcomings of Learning Management Systems(LMSs)were detected to support education in an online mode that spawned the research in Artificial Intelligence(AI)based tools that are being developed by the research community to improve the effectiveness of LMSs.This paper presents a detailed survey of the different enhancements to LMSs,which are led by key advances in the area of AI to enhance the real-time and non-real-time user experience.The AI-based enhancements proposed to the LMSs start from the Application layer and Presentation layer in the form of flipped classroom models for the efficient learning environment and appropriately designed UI/UX for efficient utilization of LMS utilities and resources,including AI-based chatbots.Session layer enhancements are also required,such as AI-based online proctoring and user authentication using Biometrics.These extend to the Transport layer to support real-time and rate adaptive encrypted video transmission for user security/privacy and satisfactory working of AI-algorithms.It also needs the support of the Networking layer for IP-based geolocation features,the Virtual Private Network(VPN)feature,and the support of Software-Defined Networks(SDN)for optimum Quality of Service(QoS).Finally,in addition to these,non-real-time user experience is enhanced by other AI-based enhancements such as Plagiarism detection algorithms and Data Analytics.

Towards Automated Assessment of Learning Management Systems in Higher Education Institutions in Zambia

Zambia like any other country in most African regions is still grappling with the dynamics of harnessing technology for the betterment of Higher Education. The onset of the Covid 19 pandemic brought a test for the preparedness of the Zambian Higher Education Institutions (HEIs) in harnessing technology for pedagogical activities. As countries worldwide switched to electronic learning during the pandemic, the same could not be said for Zambian HEIs. Zambian HEIs struggled to conduct pedagogical activities on learning management platforms. This study investigated the factors affecting the implementation and assessment of learning Management systems in Zambia’s HEIs. With its focus on assessing: 1) system features, 2) compliance with regulatory standards, 3) quality of service and 4) technology acceptance as the four key assessment areas of an LMS, this article proposed a model for assessing learning management systems in Zambian HEIs. To test the proposed model, a software tool was also developed.

Exploration of University English Teachers’Acceptance and Willingness to Use Learning Management System Data Analysis Tools

This study investigates university English teachers’acceptance and willingness to use learning management system(LMS)data analysis tools in their teaching practices.The research employs a mixed-method approach,combining quantitative surveys and qualitative interviews to understand teachers’perceptions and attitudes,and the factors influencing their adoption of LMS data analysis tools.The findings reveal that perceived usefulness,perceived ease of use,technical literacy,organizational support,and data privacy concerns significantly impact teachers’willingness to use these tools.Based on these insights,the study offers practical recommendations for educational institutions to enhance the effective adoption of LMS data analysis tools in English language teaching.

Resource management at the network edge for federated learning

Federated learning has been explored as a promising solution for training machine learning models at the network edge,without sharing private user data.With limited resources at the edge,new solutions must be developed to leverage the software and hardware resources as the existing solutions did not focus on resource management for network edge,specially for federated learning.In this paper,we describe the recent work on resource manage-ment at the edge and explore the challenges and future directions to allow the execution of federated learning at the edge.Problems such as the discovery of resources,deployment,load balancing,migration,and energy effi-ciency are discussed in the paper.

Enhancing Secure Development in Globally Distributed Software Product Lines: A Machine Learning-Powered Framework for Cyber-Resilient Ecosystems

Embracing software product lines(SPLs)is pivotal in the dynamic landscape of contemporary software devel-opment.However,the flexibility and global distribution inherent in modern systems pose significant challenges to managing SPL variability,underscoring the critical importance of robust cybersecurity measures.This paper advocates for leveraging machine learning(ML)to address variability management issues and fortify the security of SPL.In the context of the broader special issue theme on innovative cybersecurity approaches,our proposed ML-based framework offers an interdisciplinary perspective,blending insights from computing,social sciences,and business.Specifically,it employs ML for demand analysis,dynamic feature extraction,and enhanced feature selection in distributed settings,contributing to cyber-resilient ecosystems.Our experiments demonstrate the framework’s superiority,emphasizing its potential to boost productivity and security in SPLs.As digital threats evolve,this research catalyzes interdisciplinary collaborations,aligning with the special issue’s goal of breaking down academic barriers to strengthen digital ecosystems against sophisticated attacks while upholding ethics,privacy,and human values.

Survey on AI and Machine Learning Techniques for Microgrid Energy Management Systems

In the era of an energy revolution,grid decentralization has emerged as a viable solution to meet the increasing global energy demand by incorporating renewables at the distributed level.Microgrids are considered a driving component for accelerating grid decentralization.To optimally utilize the available resources and address potential challenges,there is a need to have an intelligent and reliable energy management system(EMS)for the microgrid.The artificial intelligence field has the potential to address the problems in EMS and can provide resilient,efficient,reliable,and scalable solutions.This paper presents an overview of existing conventional and AI-based techniques for energy management systems in microgrids.We analyze EMS methods for centralized,decentralized,and distributed microgrids separately.Then,we summarize machine learning techniques such as ANNs,federated learning,LSTMs,RNNs,and reinforcement learning for EMS objectives such as economic dispatch,optimal power flow,and scheduling.With the incorporation of AI,microgrids can achieve greater performance efficiency and more reliability for managing a large number of energy resources.However,challenges such as data privacy,security,scalability,explainability,etc.,need to be addressed.To conclude,the authors state the possible future research directions to explore AI-based EMS's potential in real-world applications.

A New Solution to Intrusion Detection Systems Based on Improved Federated-Learning Chain

In the context of enterprise systems,intrusion detection(ID)emerges as a critical element driving the digital transformation of enterprises.With systems spanning various sectors of enterprises geographically dispersed,the necessity for seamless information exchange has surged significantly.The existing cross-domain solutions are challenged by such issues as insufficient security,high communication overhead,and a lack of effective update mechanisms,rendering them less feasible for prolonged application on resource-limited devices.This study proposes a new cross-domain collaboration scheme based on federated chains to streamline the server-side workload.Within this framework,individual nodes solely engage in training local data and subsequently amalgamate the final model employing a federated learning algorithm to uphold enterprise systems with efficiency and security.To curtail the resource utilization of blockchains and deter malicious nodes,a node administration module predicated on the workload paradigm is introduced,enabling the release of surplus resources in response to variations in a node’s contribution metric.Upon encountering an intrusion,the system triggers an alert and logs the characteristics of the breach,facilitating a comprehensive global update across all nodes for collective defense.Experimental results across multiple scenarios have verified the security and effectiveness of the proposed solution,with no loss of its recognition accuracy.

Security Monitoring and Management for the Network Services in the Orchestration of SDN-NFV Environment Using Machine Learning Techniques

Software Defined Network(SDN)and Network Function Virtualization(NFV)technology promote several benefits to network operators,including reduced maintenance costs,increased network operational performance,simplified network lifecycle,and policies management.Network vulnerabilities try to modify services provided by Network Function Virtualization MANagement and Orchestration(NFV MANO),and malicious attacks in different scenarios disrupt the NFV Orchestrator(NFVO)and Virtualized Infrastructure Manager(VIM)lifecycle management related to network services or individual Virtualized Network Function(VNF).This paper proposes an anomaly detection mechanism that monitors threats in NFV MANO and manages promptly and adaptively to implement and handle security functions in order to enhance the quality of experience for end users.An anomaly detector investigates these identified risks and provides secure network services.It enables virtual network security functions and identifies anomalies in Kubernetes(a cloud-based platform).For training and testing purpose of the proposed approach,an intrusion-containing dataset is used that hold multiple malicious activities like a Smurf,Neptune,Teardrop,Pod,Land,IPsweep,etc.,categorized as Probing(Prob),Denial of Service(DoS),User to Root(U2R),and Remote to User(R2L)attacks.An anomaly detector is anticipated with the capabilities of a Machine Learning(ML)technique,making use of supervised learning techniques like Logistic Regression(LR),Support Vector Machine(SVM),Random Forest(RF),Naïve Bayes(NB),and Extreme Gradient Boosting(XGBoost).The proposed framework has been evaluated by deploying the identified ML algorithm on a Jupyter notebook in Kubeflow to simulate Kubernetes for validation purposes.RF classifier has shown better outcomes(99.90%accuracy)than other classifiers in detecting anomalies/intrusions in the containerized environment.

Enhanced Deep Reinforcement Learning Strategy for Energy Management in Plug-in Hybrid Electric Vehicles with Entropy Regularization and Prioritized Experience Replay

Plug-in Hybrid Electric Vehicles(PHEVs)represent an innovative breed of transportation,harnessing diverse power sources for enhanced performance.Energy management strategies(EMSs)that coordinate and control different energy sources is a critical component of PHEV control technology,directly impacting overall vehicle performance.This study proposes an improved deep reinforcement learning(DRL)-based EMSthat optimizes realtime energy allocation and coordinates the operation of multiple power sources.Conventional DRL algorithms struggle to effectively explore all possible state-action combinations within high-dimensional state and action spaces.They often fail to strike an optimal balance between exploration and exploitation,and their assumption of a static environment limits their ability to adapt to changing conditions.Moreover,these algorithms suffer from low sample efficiency.Collectively,these factors contribute to convergence difficulties,low learning efficiency,and instability.To address these challenges,the Deep Deterministic Policy Gradient(DDPG)algorithm is enhanced using entropy regularization and a summation tree-based Prioritized Experience Replay(PER)method,aiming to improve exploration performance and learning efficiency from experience samples.Additionally,the correspondingMarkovDecision Process(MDP)is established.Finally,an EMSbased on the improvedDRLmodel is presented.Comparative simulation experiments are conducted against rule-based,optimization-based,andDRL-based EMSs.The proposed strategy exhibitsminimal deviation fromthe optimal solution obtained by the dynamic programming(DP)strategy that requires global information.In the typical driving scenarios based onWorld Light Vehicle Test Cycle(WLTC)and New European Driving Cycle(NEDC),the proposed method achieved a fuel consumption of 2698.65 g and an Equivalent Fuel Consumption(EFC)of 2696.77 g.Compared to the DP strategy baseline,the proposed method improved the fuel efficiency variances(FEV)by 18.13%,15.1%,and 8.37%over the Deep QNetwork(DQN),Double DRL(DDRL),and original DDPG methods,respectively.The observational outcomes demonstrate that the proposed EMS based on improved DRL framework possesses good real-time performance,stability,and reliability,effectively optimizing vehicle economy and fuel consumption.

Optimal Machine Learning Enabled Performance Monitoring for Learning Management Systems

Learning Management System(LMS)is an application software that is used in automation,delivery,administration,tracking,and reporting of courses and programs in educational sector.The LMS which exploits machine learning(ML)has the ability of accessing user data and exploit it for improving the learning experience.The recently developed artificial intelligence(AI)and ML models helps to accomplish effective performance monitoring for LMS.Among the different processes involved in ML based LMS,feature selection and classification processesfind beneficial.In this motivation,this study introduces Glowworm-based Feature Selection with Machine Learning Enabled Performance Monitoring(GSO-MFWELM)technique for LMS.The key objective of the proposed GSO-MFWELM technique is to effectually monitor the performance in LMS.The pro-posed GSO-MFWELM technique involves GSO-based feature selection techni-que to select the optimal features.Besides,Weighted Extreme Learning Machine(WELM)model is applied for classification process whereas the parameters involved in WELM model are optimallyfine-tuned with the help of May-fly Optimization(MFO)algorithm.The design of GSO and MFO techniques result in reduced computation complexity and improved classification performance.The presented GSO-MFWELM technique was validated for its performance against benchmark dataset and the results were inspected under several aspects.The simulation results established the supremacy of GSO-MFWELM technique over recent approaches with the maximum classification accuracy of 0.9589.

Intelligent Student Mental Health Assessment Model on Learning Management System

A learning management system(LMS)is a software or web based application,commonly utilized for planning,designing,and assessing a particular learning procedure.Generally,the LMS offers a method of creating and delivering content to the instructor,monitoring students’involvement,and validating their outcomes.Since mental health issues become common among studies in higher education globally,it is needed to properly determine it to improve mental stabi-lity.This article develops a new seven spot lady bird feature selection with opti-mal sparse autoencoder(SSLBFS-OSAE)model to assess students’mental health on LMS.The major aim of the SSLBFS-OSAE model is to determine the proper health status of the students with respect to depression,anxiety,and stress(DAS).The SSLBFS-OSAE model involves a new SSLBFS model to elect a useful set of features.In addition,OSAE model is applied for the classification of mental health conditions and the performance can be improved by the use of cuckoo search optimization(CSO)based parameter tuning process.The design of CSO algorithm for optimally tuning the SAE parameters results in enhanced classifica-tion outcomes.For examining the improved classifier results of the SSLBFS-OSAE model,a comprehensive results analysis is done and the obtained values highlighted the supremacy of the SSLBFS model over its recent methods interms of different measures.

Optimization of Interactive Videos Empowered the Experience of Learning Management System

The Learning management system(LMS)is now being used for uploading educational content in both distance and blended setups.LMS platform has two types of users:the educators who upload the content,and the students who have to access the content.The students,usually rely on text notes or books and video tutorials while their exams are conducted with formal methods.Formal assessments and examination criteria are ineffective with restricted learning space which makes the student tend only to read the educational contents and videos instead of interactive mode.The aim is to design an interactive LMS and examination video-based interface to cater the issues of educators and students.It is designed according to Human-computer interaction(HCI)principles to make the interactive User interface(UI)through User experience(UX).The interactive lectures in the form of annotated videos increase user engagement and improve the self-study context of users involved in LMS.The interface design defines how the design will interact with users and how the interface exchanges information.The findings show that interactive videos for LMS allow the users to have a more personalized learning experience by engaging in the educational content.The result shows a highly personalized learning experience due to the interactive video and quiz within the video.

Exploring Students Engagement Towards the Learning Management System (LMS) Using Learning Analytics

Learning analytics is a rapidly evolving research discipline that uses theinsights generated from data analysis to support learners as well as optimize boththe learning process and environment. This paper studied students’ engagementlevel of the Learning Management System (LMS) via a learning analytics tool,student’s approach in managing their studies and possible learning analytic methods to analyze student data. Moreover, extensive systematic literature review(SLR) was employed for the selection, sorting and exclusion of articles fromdiverse renowned sources. The findings show that most of the engagement inLMS are driven by educators. Additionally, we have discussed the factors inLMS, causes of low engagement and ways of increasing engagement factorsvia the Learning Analytics approach. Nevertheless, apart from recognizing theLearning Analytics approach as being a successful method and technique for analyzing the LMS data, this research further highlighted the possibility of mergingthe learning analytics technique with the LMS engagement in every institution asbeing a direction for future research.

Reinforcement Learning-Based Electric Vehicles Energy Management Strategy with Battery Thermal Model

The promotion of electric vehicles(EVs)is restricted due to their short cruising range.It is desirable to design an effective energy management strategy to improve their energy efficiency.Most existing work concerning energy management strategies focused on hybrids rather than the EVs.The work focusing on the energy management strategy for EVs mainly uses the traditional optimization strategies,thereby limiting the advantages of energy economy.To this end,a novel energy management strategy that considered the impact of battery thermal effects was proposed with the help of reinforcement learning.The main idea was to first analyze the energy flow path of EVs,further formulize the energy management as an optimization problem,and finally propose an online strategy based on reinforcement learning to obtain the optimal strategy.Additionally,extensive simulation results have demonstrated that our strategy reduces energy consumption by at least 27.4%compared to the existing methods.

Systematic review of data-centric approaches in artificial intelligence and machine learning

Artificial intelligence(AI)relies on data and algorithms.State-of-the-art(SOTA)AI smart algorithms have been developed to improve the performance of AI-oriented structures.However,model-centric approaches are limited by the absence of high-quality data.Data-centric AI is an emerging approach for solving machine learning(ML)problems.It is a collection of various data manipulation techniques that allow ML practitioners to systematically improve the quality of the data used in an ML pipeline.However,data-centric AI approaches are not well documented.Researchers have conducted various experiments without a clear set of guidelines.This survey highlights six major data-centric AI aspects that researchers are already using to intentionally or unintentionally improve the quality of AI systems.These include big data quality assessment,data preprocessing,transfer learning,semi-supervised learning,machine learning operations(MLOps),and the effect of adding more data.In addition,it highlights recent data-centric techniques adopted by ML practitioners.We addressed how adding data might harm datasets and how HoloClean can be used to restore and clean them.Finally,we discuss the causes of technical debt in AI.Technical debt builds up when software design and implementation decisions run into“or outright collide with”business goals and timelines.This survey lays the groundwork for future data-centric AI discussions by summarizing various data-centric approaches.

Characterization of Memory Access in Deep Learning and Its Implications in Memory Management

Due to the recent trend of software intelligence in the Fourth Industrial Revolution,deep learning has become a mainstream workload for modern computer systems.Since the data size of deep learning increasingly grows,managing the limited memory capacity efficiently for deep learning workloads becomes important.In this paper,we analyze memory accesses in deep learning workloads and find out some unique characteristics differentiated from traditional workloads.First,when comparing instruction and data accesses,data access accounts for 96%–99%of total memory accesses in deep learning workloads,which is quite different from traditional workloads.Second,when comparing read and write accesses,write access dominates,accounting for 64%–80%of total memory accesses.Third,although write access makes up the majority of memory accesses,it shows a low access bias of 0.3 in the Zipf parameter.Fourth,in predicting re-access,recency is important in read access,but frequency provides more accurate information in write access.Based on these observations,we introduce a Non-Volatile Random Access Memory(NVRAM)-accelerated memory architecture for deep learning workloads,and present a new memory management policy for this architecture.By considering the memory access characteristics of deep learning workloads,the proposed policy improves memory performance by 64.3%on average compared to the CLOCK policy.

Privacy Data Management Mechanism Based on Blockchain and Federated Learning

Due to the extensive use of various intelligent terminals and the popularity of network social tools,a large amount of data in the field of medical emerged.How to manage these massive data safely and reliably has become an important challenge for the medical network community.This paper proposes a data management framework of medical network community based on Consortium Blockchain(CB)and Federated learning(FL),which realizes the data security sharing between medical institutions and research institutions.Under this framework,the data security sharing mechanism of medical network community based on smart contract and the data privacy protection mechanism based on FL and alliance chain are designed to ensure the security of data and the privacy of important data in medical network community,respectively.An intelligent contract system based on Keyed-Homomorphic Public Key(KH-PKE)Encryption scheme is designed,so that medical data can be saved in the CB in the form of ciphertext,and the automatic sharing of data is realized.Zero knowledge mechanism is used to ensure the correctness of shared data.Moreover,the zero-knowledge mechanism introduces the dynamic group signature mechanism of chosen ciphertext attack(CCA)anonymity,which makes the scheme more efficient in computing and communication cost.In the end of this paper,the performance of the scheme is analyzed fromboth asymptotic and practical aspects.Through experimental comparative analysis,the scheme proposed in this paper is more effective and feasible.