Filter Bank Networks for Few-Shot Class-Incremental Learning

Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the dynamically changing world,e.g.,classifying newly discovered fish species,remains an open problem.We address an even more challenging and realistic setting of this problem where new class samples are insufficient,i.e.,Few-Shot Class-Incremental Learning(FSCIL).Current FSCIL methods augment the training data to alleviate the overfitting of novel classes.By contrast,we propose Filter Bank Networks(FBNs)that augment the learnable filters to capture fine-detailed features for adapting to future new classes.In the forward pass,FBNs augment each convolutional filter to a virtual filter bank containing the canonical one,i.e.,itself,and multiple transformed versions.During back-propagation,FBNs explicitly stimulate fine-detailed features to emerge and collectively align all gradients of each filter bank to learn the canonical one.FBNs capture pattern variants that do not yet exist in the pretraining session,thus making it easy to incorporate new classes in the incremental learning phase.Moreover,FBNs introduce model-level prior knowledge to efficiently utilize the limited few-shot data.Extensive experiments on MNIST,CIFAR100,CUB200,andMini-ImageNet datasets show that FBNs consistently outperformthe baseline by a significantmargin,reporting new state-of-the-art FSCIL results.In addition,we contribute a challenging FSCIL benchmark,Fishshot1K,which contains 8261 underwater images covering 1000 ocean fish species.The code is included in the supplementary materials.

ILIDViz:An incremental learning-based visual analysis system for network anomaly detection

Background With the development of information technology,there is a significant increase in the number of network traffic logs mixed with various types of cyberattacks.Traditional intrusion detection systems(IDSs)are limited in detecting new inconstant patterns and identifying malicious traffic traces in real time.Therefore,there is an urgent need to implement more effective intrusion detection technologies to protect computer security.Methods In this study,we designed a hybrid IDS by combining our incremental learning model(KANSOINN)and active learning to learn new log patterns and detect various network anomalies in real time.Conclusions Experimental results on the NSLKDD dataset showed that KAN-SOINN can be continuously improved and effectively detect malicious logs.Meanwhile,comparative experiments proved that using a hybrid query strategy in active learning can improve the model learning efficiency.

Incremental Learning Based on Data Translation and Knowledge Distillation

Recently, deep convolutional neural networks (DCNNs) have achieved remarkable results in image classification tasks. Despite convolutional networks’ great successes, their training process relies on a large amount of data prepared in advance, which is often challenging in real-world applications, such as streaming data and concept drift. For this reason, incremental learning (continual learning) has attracted increasing attention from scholars. However, incremental learning is associated with the challenge of catastrophic forgetting: the performance on previous tasks drastically degrades after learning a new task. In this paper, we propose a new strategy to alleviate catastrophic forgetting when neural networks are trained in continual domains. Specifically, two components are applied: data translation based on transfer learning and knowledge distillation. The former translates a portion of new data to reconstruct the partial data distribution of the old domain. The latter uses an old model as a teacher to guide a new model. The experimental results on three datasets have shown that our work can effectively alleviate catastrophic forgetting by a combination of the two methods aforementioned.

Incremental Learning Model for Load Forecasting without Training Sample

This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.However,the use of OS-ELM requires a sufficient amount of initial training sample data,which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained.To solve this problem,a synthesis of the initial training sample is proposed.The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples.Then the synthesis samples are used to initial train the OS-ELM.This proposed method is compared with Fully Online Extreme Learning Machine(FOS-ELM),which is an incremental learning model that also does not require the initial training samples.Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset.Experiments have shown that the proposed method with a wide range of noise levels,can forecast hourly load more accurately than the FOS-ELM.

Incremental Learning Framework for Mining Big Data Stream

At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data,it also supposes that all data are pre-classified.Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features.The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model.The proposed model is implemented in spark architecture.For each data stream,the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation.This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency.The results show that the proposed model outperforms stateof-the-art performance in terms of accuracy(0.861)precision(0.9328)and minimal MSE(0.0416).

Intrusion Detection Method Based on Active Incremental Learning in Industrial Internet of Things Environment

Intrusion detection is a hot field in the direction of network security.Classical intrusion detection systems are usually based on supervised machine learning models.These offline-trained models usually have better performance in the initial stages of system construction.However,due to the diversity and rapid development of intrusion techniques,the trained models are often difficult to detect new attacks.In addition,very little noisy data in the training process often has a considerable impact on the performance of the intrusion detection system.This paper proposes an intrusion detection system based on active incremental learning with the adaptive capability to solve these problems.IDS consists of two modules,namely the improved incremental stacking ensemble learning detection method called Multi-Stacking model and the active learning query module.The stacking model can cope well with concept drift due to the diversity and generalization selection of its base classifiers,but the accuracy does not meet the requirements.The Multi-Stacking model improves the accuracy of the model by adding a voting layer on the basis of the original stacking.The active learning query module improves the detection of known attacks through the committee algorithm,and the improved KNN algorithm can better help detect unknown attacks.We have tested the latest industrial IoT dataset with satisfactory results.

Selective and Adaptive Incremental Transfer Learning with Multiple Datasets for Machine Fault Diagnosis

The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation for automatically recognizing machine failure,and thus timely maintenance can ensure safe operations.Transfer learning is a promising solution that can enhance the machine fault diagnosis model by borrowing pre-trained knowledge from the source model and applying it to the target model,which typically involves two datasets.In response to the availability of multiple datasets,this paper proposes using selective and adaptive incremental transfer learning(SA-ITL),which fuses three algorithms,namely,the hybrid selective algorithm,the transferability enhancement algorithm,and the incremental transfer learning algorithm.It is a selective algorithm that enables selecting and ordering appropriate datasets for transfer learning and selecting useful knowledge to avoid negative transfer.The algorithm also adaptively adjusts the portion of training data to balance the learning rate and training time.The proposed algorithm is evaluated and analyzed using ten benchmark datasets.Compared with other algorithms from existing works,SA-ITL improves the accuracy of all datasets.Ablation studies present the accuracy enhancements of the SA-ITL,including the hybrid selective algorithm(1.22%-3.82%),transferability enhancement algorithm(1.91%-4.15%),and incremental transfer learning algorithm(0.605%-2.68%).These also show the benefits of enhancing the target model with heterogeneous image datasets that widen the range of domain selection between source and target domains.

Ethical Decision-Making Framework Based on Incremental ILP Considering Conflicts

Humans are experiencing the inclusion of artificial agents in their lives,such as unmanned vehicles,service robots,voice assistants,and intelligent medical care.If the artificial agents cannot align with social values or make ethical decisions,they may not meet the expectations of humans.Traditionally,an ethical decision-making framework is constructed by rule-based or statistical approaches.In this paper,we propose an ethical decision-making framework based on incremental ILP(Inductive Logic Programming),which can overcome the brittleness of rule-based approaches and little interpretability of statistical approaches.As the current incremental ILP makes it difficult to solve conflicts,we propose a novel ethical decision-making framework considering conflicts in this paper,which adopts our proposed incremental ILP system.The framework consists of two processes:the learning process and the deduction process.The first process records bottom clauses with their score functions and learns rules guided by the entailment and the score function.The second process obtains an ethical decision based on the rules.In an ethical scenario about chatbots for teenagers’mental health,we verify that our framework can learn ethical rules and make ethical decisions.Besides,we extract incremental ILP from the framework and compare it with the state-of-the-art ILP systems based on ASP(Answer Set Programming)focusing on conflict resolution.The results of comparisons show that our proposed system can generate better-quality rules than most other systems.

Adaptive Spectral Clustering Ensemble Selection via Resampling and Population-Based Incremental Learning Algorithm

In this paper, we explore a novel ensemble method for spectral clustering. In contrast to the traditional clustering ensemble methods that combine all the obtained clustering results, we propose the adaptive spectral clustering ensemble method to achieve a better clustering solution. This method can adaptively assess the number of the component members, which is not owned by many other algorithms. The component clusterings of the ensemble system are generated by spectral clustering （SC） which bears some good characteristics to engender the diverse committees. The selection process works by evaluating the generated component spectral clustering through resampling technique and population-based incremental learning algorithm （PBIL）. Experimental results on UCI datasets demonstrate that the proposed algorithm can achieve better results compared with traditional clustering ensemble methods, especially when the number of component clusterings is large.

Incremental support vector machine algorithm based on multi-kernel learning

A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.

An artificial immune and incremental learning inspired novel framework for performance pattern identification of complex electromechanical systems

Performance pattern identification is the key basis for fault detection and condition prediction,which plays a major role in ensuring safety and reliability in complex electromechanical systems(CESs).However,there are a few problems related to the automatic and adaptive updating of an identification model.Aiming to solve the problem of identification model updating,a novel framework for performance pattern identification of the CESs based on the artificial immune systems and incremental learning is proposed in this paper to classify real-time monitoring data into different performance patterns.First,an unsupervised clustering technique is used to construct an initial identification model.Second,the artificial immune and outlier detection algorithms are applied to identify abnormal data and determine the type of immune response.Third,incremental learning is employed to trace the dynamic changes of patterns,and operations such as pattern insertion,pattern removal,and pattern revision are designed to realize automatic and adaptive updates of an identification model.The effectiveness of the proposed framework is demonstrated through experiments with the benchmark and actual pattern identification applications.As an unsupervised and self-adapting approach,the proposed framework inherits the preponderances of the conventional methods but overcomes some of their drawbacks because the retraining process is not required in perceiving the pattern changes.Therefore,this method can be flexibly and efficiently used for performance pattern identification of the CESs.Moreover,the proposed method provides a foundation for fault detection and condition prediction,and can be used in other engineering applications.

OKO-SVM:Online kernel optimization-based support vector machine for the incremental learning and classification of the sentiments in the train reviews

Online incremental learning is one of the emerging research interests among the researchers in the recent years.The sentiment classification through the online incremental learning faces many challenges due to the limitations in the memory and the computing resources available for processing the online reviews.This work has introduced an online incremental learning algorithm for classifying the train reviews.The sentiments available in the reviews provided for the public services are necessary for improving the quality of the service.This work proposes the online kernel optimizationbased support vector machine(OKO-SVM)classifier for the sentiment classification of the train reviews.This paper is the extension of the previous work kernel optimizationbased support vector machine(KO-SVM).The OKO-SVM classifier uses the proposed fuzzy bound for modifying the weight for each incoming review database for the particular time duration.The simulation uses the standard train review and the movie review database for the classification.From the simulation results,it is evident that the proposed model has achieved a better performance with the values of 84.42%,93.86%,and 74.56%regarding the accuracy,sensitivity,and specificity while classifying the train review database.

Incremental POP Learning

In recently proposed partial oblique projection (POP) learning, a function space is decomposed into two complementary subspaces, so that functions belonging to one of which can be optimally estimated. This paper shows that when the decomposition is specially performed so that the above subspace becomes the largest, a special learning called SPOP learning is obtained and correspondingly an incremental learning is implemented, result of which equals exactly to that of batch learning including novel data. The effectiveness of the method is illustrated by experimental results.

A Super-resolution Perception-based Incremental Learning Approach for Power System Voltage Stability Assessment with Incomplete PMU Measurements

This paper develops a fully data-driven,missingdata tolerant method for post-fault short-term voltage stability(STVS)assessment of power systems against the incomplete PMU measurements.The super-resolution perception(SRP),based on a deep residual learning convolutional neural network,is employed to cope with the missing PMU measurements.The incremental broad learning(BL)is used to rapidly update the model to maintain and enhance the online application performance.Being different from the state-of-the-art methods,the proposed method is fully data-driven and can fill up missing data under any PMU placement information loss and network topology change scenario.Simulation results demonstrate that the proposed method has the best performance in terms of STVS assessment accuracy and missing-data tolerance among the existing methods on the benchmark testing system.

Unknown DDoS Attack Detection with Fuzzy C-Means Clustering and Spatial Location Constraint Prototype Loss

Since its inception,the Internet has been rapidly evolving.With the advancement of science and technology and the explosive growth of the population,the demand for the Internet has been on the rise.Many applications in education,healthcare,entertainment,science,and more are being increasingly deployed based on the internet.Concurrently,malicious threats on the internet are on the rise as well.Distributed Denial of Service(DDoS)attacks are among the most common and dangerous threats on the internet today.The scale and complexity of DDoS attacks are constantly growing.Intrusion Detection Systems(IDS)have been deployed and have demonstrated their effectiveness in defense against those threats.In addition,the research of Machine Learning(ML)and Deep Learning(DL)in IDS has gained effective results and significant attention.However,one of the challenges when applying ML and DL techniques in intrusion detection is the identification of unknown attacks.These attacks,which are not encountered during the system’s training,can lead to misclassification with significant errors.In this research,we focused on addressing the issue of Unknown Attack Detection,combining two methods:Spatial Location Constraint Prototype Loss(SLCPL)and Fuzzy C-Means(FCM).With the proposed method,we achieved promising results compared to traditional methods.The proposed method demonstrates a very high accuracy of up to 99.8%with a low false positive rate for known attacks on the Intrusion Detection Evaluation Dataset(CICIDS2017)dataset.Particularly,the accuracy is also very high,reaching 99.7%,and the precision goes up to 99.9%for unknown DDoS attacks on the DDoS Evaluation Dataset(CICDDoS2019)dataset.The success of the proposed method is due to the combination of SLCPL,an advanced Open-Set Recognition(OSR)technique,and FCM,a traditional yet highly applicable clustering technique.This has yielded a novel method in the field of unknown attack detection.This further expands the trend of applying DL and ML techniques in the development of intrusion detection systems and cybersecurity.Finally,implementing the proposed method in real-world systems can enhance the security capabilities against increasingly complex threats on computer networks.

An improved transfer learning strategy for short-term cross-building energy prediction usingdata incremental

The available modelling data shortage issue makes it difficult to guarantee the performance of data-driven building energy prediction(BEP)models for both the newly built buildings and existing information-poor buildings.Both knowledge transfer learning(KTL)and data incremental learning(DIL)can address the data shortage issue of such buildings.For new building scenarios with continuous data accumulation,the performance of BEP models has not been fully investigated considering the data accumulation dynamics.DIL,which can learn dynamic features from accumulated data adapting to the developing trend of new building time-series data and extend BEP model's knowledge,has been rarely studied.Previous studies have shown that the performance of KTL models trained with fixed data can be further improved in scenarios with dynamically changing data.Hence,this study proposes an improved transfer learning cross-BEP strategy continuously updated using the coarse data incremental(CDI)manner.The hybrid KTL-DIL strategy(LSTM-DANN-CDI)uses domain adversarial neural network(DANN)for KLT and long short-term memory(LSTM)as the Baseline BEP model.Performance evaluation is conducted to systematically qualify the effectiveness and applicability of KTL and improved KTL-DIL.Real-world data from six-type 36 buildings of six types are adopted to evaluate the performance of KTL and KTL-DIL in data-driven BEP tasks considering factors like the model increment time interval,the available target and source building data volumes.Compared with LSTM,results indicate that KTL(LSTM-DANN)and the proposed KTL-DIL(LSTM-DANN-CDI)can significantly improve the BEP performance for new buildings with limited data.Compared with the pure KTL strategy LSTM-DANN,the improved KTL-DIL strategy LSTM-DANN-CDI has better prediction performance with an average performance improvement ratio of 60%.

Incremental Data Stream Classification with Adaptive Multi-Task Multi-View Learning

With the enhancement of data collection capabilities,massive streaming data have been accumulated in numerous application scenarios.Specifically,the issue of classifying data streams based on mobile sensors can be formalized as a multi-task multi-view learning problem with a specific task comprising multiple views with shared features collected from multiple sensors.Existing incremental learning methods are often single-task single-view,which cannot learn shared representations between relevant tasks and views.An adaptive multi-task multi-view incremental learning framework for data stream classification called MTMVIS is proposed to address the above challenges,utilizing the idea of multi-task multi-view learning.Specifically,the attention mechanism is first used to align different sensor data of different views.In addition,MTMVIS uses adaptive Fisher regularization from the perspective of multi-task multi-view learning to overcome catastrophic forgetting in incremental learning.Results reveal that the proposed framework outperforms state-of-the-art methods based on the experiments on two different datasets with other baselines.

Research on the IL-Bagging-DHKELM Short-Term Wind Power Prediction Algorithm Based on Error AP Clustering Analysis

With the continuous advancement of China’s“peak carbon dioxide emissions and Carbon Neutrality”process,the proportion of wind power is increasing.In the current research,aiming at the problem that the forecasting model is outdated due to the continuous updating of wind power data,a short-term wind power forecasting algorithm based on Incremental Learning-Bagging Deep Hybrid Kernel Extreme Learning Machine(IL-Bagging-DHKELM)error affinity propagation cluster analysis is proposed.The algorithm effectively combines deep hybrid kernel extreme learning machine(DHKELM)with incremental learning(IL).Firstly,an initial wind power prediction model is trained using the Bagging-DHKELM model.Secondly,Euclidean morphological distance affinity propagation AP clustering algorithm is used to cluster and analyze the prediction error of wind power obtained from the initial training model.Finally,the correlation between wind power prediction errors and Numerical Weather Prediction(NWP)data is introduced as incremental updates to the initial wind power prediction model.During the incremental learning process,multiple error performance indicators are used to measure the overall model performance,thereby enabling incremental updates of wind power models.Practical examples show the method proposed in this article reduces the root mean square error of the initial model by 1.9 percentage points,indicating that this method can be better adapted to the current scenario of the continuous increase in wind power penetration rate.The accuracy and precision of wind power generation prediction are effectively improved through the method.

Self-Care Assessment for Daily Living Using Machine Learning Mechanism

Nowadays,activities of daily living(ADL)recognition system has been considered an important field of computer vision.Wearable and optical sensors are widely used to assess the daily living activities in healthy people and people with certain disorders.Although conventional ADL utilizes RGB optical sensors but an RGB-D camera with features of identifying depth(distance information)and visual cues has greatly enhanced the performance of activity recognition.In this paper,an RGB-D-based ADL recognition system has been presented.Initially,human silhouette has been extracted from the noisy background of RGB and depth images to track human movement in a scene.Based on these silhouettes,full body features and point based features have been extracted which are further optimized with probability based incremental learning(PBIL)algorithm.Finally,random forest classifier has been used to classify activities into different categories.The n-fold crossvalidation scheme has been used to measure the viability of the proposed model on the RGBD-AC benchmark dataset and has achieved an accuracy of 92.71%over other state-of-the-art methodologies.

An Online Chronic Disease Prediction System Based on Incremental Deep Neural Network

Many chronic disease prediction methods have been proposed to predict or evaluate diabetes through artificial neural network.However,due to the complexity of the human body,there are still many challenges to face in that process.One of them is how to make the neural network prediction model continuously adapt and learn disease data of different patients,online.This paper presents a novel chronic disease prediction system based on an incremental deep neural network.The propensity of users suffering from chronic diseases can continuously be evaluated in an incremental manner.With time,the system can predict diabetes more and more accurately by processing the feedback information.Many diabetes prediction studies are based on a common dataset,the Pima Indians diabetes dataset,which has only eight input attributes.In order to determine the correlation between the pathological characteristics of diabetic patients and their daily living resources,we have established an in-depth cooperation with a hospital.A Chinese diabetes dataset with 575 diabetics was created.Users’data collected by different sensors were used to train the network model.We evaluated our system using a real-world diabetes dataset to confirm its effectiveness.The experimental results show that the proposed system can not only continuously monitor the users,but also give early warning of physiological data that may indicate future diabetic ailments.