Hyperparameter Optimization for Capsule Network Based Modified Hybrid Rice Optimization Algorithm

Hyperparameters play a vital impact in the performance of most machine learning algorithms.It is a challenge for traditional methods to con-figure hyperparameters of the capsule network to obtain high-performance manually.Some swarm intelligence or evolutionary computation algorithms have been effectively employed to seek optimal hyperparameters as a com-binatorial optimization problem.However,these algorithms are prone to get trapped in the local optimal solution as random search strategies are adopted.The inspiration for the hybrid rice optimization(HRO)algorithm is from the breeding technology of three-line hybrid rice in China,which has the advantages of easy implementation,less parameters and fast convergence.In the paper,genetic search is combined with the hybrid rice optimization algorithm(GHRO)and employed to obtain the optimal hyperparameter of the capsule network automatically,that is,a probability search technique and a hybridization strategy belong with the primary HRO.Thirteen benchmark functions are used to evaluate the performance of GHRO.Furthermore,the MNIST,Chest X-Ray(pneumonia),and Chest X-Ray(COVID-19&pneumonia)datasets are also utilized to evaluate the capsule network learnt by GHRO.The experimental results show that GHRO is an effective method for optimizing the hyperparameters of the capsule network,which is able to boost the performance of the capsule network on image classification.

Effective Customer Review Analysis Using Combined Capsule Networks with Matrix Factorization Filtering

Nowadays,commercial transactions and customer reviews are part of human life and various business applications.The technologies create a great impact on online user reviews and activities,affecting the business process.Customer reviews and ratings are more helpful to the new customer to purchase the product,but the fake reviews completely affect the business.The traditional systems consume maximum time and create complexity while analyzing a large volume of customer information.Therefore,in this work optimized recommendation system is developed for analyzing customer reviews with minimum complexity.Here,Amazon Product Kaggle dataset information is utilized for investigating the customer review.The collected information is analyzed and processed by batch normalized capsule networks(NCN).The network explores the user reviews according to product details,time,price purchasing factors,etc.,ensuring product quality and ratings.Then effective recommendation system is developed using a butterfly optimized matrix factorizationfiltering approach.Then the system’s efficiency is evaluated using the Rand Index,Dunn index,accuracy,and error rate.

Rolling Bearing Fault Diagnosis Based On Convolutional Capsule Network

Fault diagnosis technology has been widely applied and is an important part of ensuring the safe operation of mechanical equipment.In response to the problem of frequent faults in rolling bearings,this paper designs a rolling bearing fault diagnosis method based on convolutional capsule network(CCN).More specifically,the original vibration signal is converted into a two-dimensional time–frequency image using continuous wavelet transform(CWT),and the feature extraction is performed on the two-dimensional time–frequency image using the convolution layer at the front end of the network,and the extracted features are input into the capsule network.The capsule network converts the extracted features into vector neurons,and the dynamic routing algorithm is used to achieve feature transfer and output the results of fault diagnosis.Two different datasets are used to compare with other traditional deep learning models to verify the fault diagnosis capability of the method.The results show that the CCN has good diagnostic capability under different working conditions,even in the presence of noise and insufficient samples,compared to other models.This method contributes to the safe and reliable operation of mechanical equipment and is suitable for other rotating scenarios.

Using Capsule Networks for Android Malware Detection Through Orientation-Based Features

Mobile phones are an essential part of modern life.The two popular mobile phone platforms,Android and iPhone Operating System(iOS),have an immense impact on the lives of millions of people.Among these two,Android currently boasts more than 84%market share.Thus,any personal data put on it are at great risk if not properly protected.On the other hand,more than a million pieces of malware have been reported on Android in just 2021 till date.Detecting and mitigating all this malware is extremely difficult for any set of human experts.Due to this reason,machine learning-and specifically deep learning-has been utilized in the recent past to resolve this issue.However,deep learning models have primarily been designed for image analysis.While this line of research has shown promising results,it has been difficult to really understand what the features extracted by deep learning models are in the domain of malware.Moreover,due to the translation invariance property of popular models based on ConvolutionalNeural Network(CNN),the true potential of deep learning for malware analysis is yet to be realized.To resolve this issue,we envision the use of Capsule Networks(CapsNets),a state-of-the-art model in deep learning.We argue that since CapsNets are orientation-based in terms of images,they can potentially be used to capture spatial relationships between different features at different locations within a sequence of opcodes.We design a deep learning-based architecture that efficiently and effectively handles very large scale malware datasets to detect Androidmalware without resorting to very deep networks.This leads tomuch faster detection as well as increased accuracy.We achieve state-of-the-art F1 score of 0.987 with an FPR of just 0.002 for three very large,real-world malware datasets.Our code is made available as open source and can be used to further enhance our work with minimal effort.

Expression Recognition Method Based on Convolutional Neural Network and Capsule Neural Network

Convolutional neural networks struggle to accurately handle changes in angles and twists in the direction of images,which affects their ability to recognize patterns based on internal feature levels. In contrast, CapsNet overcomesthese limitations by vectorizing information through increased directionality and magnitude, ensuring that spatialinformation is not overlooked. Therefore, this study proposes a novel expression recognition technique calledCAPSULE-VGG, which combines the strengths of CapsNet and convolutional neural networks. By refining andintegrating features extracted by a convolutional neural network before introducing theminto CapsNet, ourmodelenhances facial recognition capabilities. Compared to traditional neural network models, our approach offersfaster training pace, improved convergence speed, and higher accuracy rates approaching stability. Experimentalresults demonstrate that our method achieves recognition rates of 74.14% for the FER2013 expression dataset and99.85% for the CK+ expression dataset. By contrasting these findings with those obtained using conventionalexpression recognition techniques and incorporating CapsNet’s advantages, we effectively address issues associatedwith convolutional neural networks while increasing expression identification accuracy.

Ensemble of Two-Path Capsule Networks for Diagnosis of Turner Syndrome Using Global-Local Facial Images

Turner syndrome(TS)is a chromosomal disorder disease that only affects the growth of female patients.Prompt diagnosis is of high significance for the patients.However,clinical screening methods are time-consuming and cost-expensive.Some researchers used machine learning-based methods to detect TS,the performance of which needed to be improved.Therefore,we propose an ensemble method of two-path capsule networks(CapsNets)for detecting TS based on global-local facial images.Specifically,the TS facial images are preprocessed and segmented into eight local parts under the direction of physicians;then,nine two-path CapsNets are respectively trained using the complete TS facial images and eight local images,in which the few-shot learning is utilized to solve the problem of limited data;finally,a probability-based ensemble method is exploited to combine nine classifiers for the classification of TS.By studying base classifiers,we find two meaningful facial areas are more related to TS patients,i.e.,the parts of eyes and nose.The results demonstrate that the proposed model is effective for the TS classification task,which achieves the highest accuracy of 0.9241.

Deep Capsule Residual Networks for Better Diagnosis Rate in Medical Noisy Images

With the advent of Machine and Deep Learning algorithms,medical image diagnosis has a new perception of diagnosis and clinical treatment.Regret-tably,medical images are more susceptible to capturing noises despite the peak in intelligent imaging techniques.However,the presence of noise images degrades both the diagnosis and clinical treatment processes.The existing intelligent meth-ods suffer from the deficiency in handling the diverse range of noise in the ver-satile medical images.This paper proposes a novel deep learning network which learns from the substantial extent of noise in medical data samples to alle-viate this challenge.The proposed deep learning architecture exploits the advan-tages of the capsule network,which is used to extract correlation features and combine them with redefined residual features.Additionally,thefinal stage of dense learning is replaced with powerful extreme learning machines to achieve a better diagnosis rate,even for noisy and complex images.Extensive experimen-tation has been conducted using different medical images.Various performances such as Peak-Signal-To-Noise Ratio(PSNR)and Structural-Similarity-Index-Metrics(SSIM)are compared with the existing deep learning architectures.Addi-tionally,a comprehensive analysis of individual algorithms is analyzed.The experimental results prove that the proposed model has outperformed the other existing algorithms by a substantial margin and proved its supremacy over the other learning models.

Capsule networks embedded with prior known support information for image reconstruction

Compressed sensing(CS)has been successfully applied to realize image reconstruction.Neural networks have been introduced to the CS of images to exploit the prior known support information,which can improve the reconstruction quality.Capsule Network(Caps Net)is the latest achievement in neural networks,and can well represent the instantiation parameters of a specific type of entity or part of an object.This study aims to propose a Caps Net with a novel dynamic routing to embed the information within the CS framework.The output of the network represents the probability that the index of the nonzero entry exists on the support of the signal of interest.To lead the dynamic routing to the most likely index,a group of prediction vectors is designed determined by the information.Furthermore,the results of experiments on imaging signals are taken for a comparation of the performances among different algorithms.It is concluded that the proposed capsule network(Caps Net)creates higher reconstruction quality at nearly the same time with traditional Caps Net.

Deep Capsule Residual Networks for Better Diagnosis Rate in Medical Noisy Images

With the advent of Machine and Deep Learning algorithms,medical image diagnosis has a new perception of diagnosis and clinical treatment.Regret-tably,medical images are more susceptible to capturing noises despite the peak in intelligent imaging techniques.However,the presence of noise images degrades both the diagnosis and clinical treatment processes.The existing intelligent meth-ods suffer from the deficiency in handling the diverse range of noise in the ver-satile medical images.This paper proposes a novel deep learning network which learns from the substantial extent of noise in medical data samples to alle-viate this challenge.The proposed deep learning architecture exploits the advan-tages of the capsule network,which is used to extract correlation features and combine them with redefined residual features.Additionally,the final stage of dense learning is replaced with powerful extreme learning machines to achieve a better diagnosis rate,even for noisy and complex images.Extensive experimen-tation has been conducted using different medical images.Various performances such as Peak-Signal-To-Noise Ratio(PSNR)and Structural-Similarity-Index-Metrics(SSIM)are compared with the existing deep learning architectures.Addi-tionally,a comprehensive analysis of individual algorithms is analyzed.The experimental results prove that the proposed model has outperformed the other existing algorithms by a substantial margin and proved its supremacy over the other learning models.

Fault Classification for On-board Equipment of High-speed Railway Based on Attention Capsule Network

The conventional troubleshooting methods for high-speed railway on-board equipment, with over-reliance on personnel experience, is characterized by one-sidedness and low efficiency. In the process of high-speed train operation, numerous text-based onboard logs are recorded by on-board computers. Machine learning methods can help technicians make a correct judgment of fault types using the on-board log reasonably. Therefore, a fault classification model of on-board equipment based on attention capsule networks is proposed. This paper presents an empirical exploration of the application of a capsule network with dynamic routing in fault classification. A capsule network can encode the internal spatial part-whole relationship between various entities to identify the fault types. As the importance of each word in the on-board log and the dependencies between them have a significant impact on fault classification, an attention mechanism is incorporated into the capsule network to distill important information. Considering the imbalanced distribution of normal data and fault data in the on-board log, the focal loss function is introduced into the model to adjust the imbalanced data. The experiments are conducted on the on-board log of a railway bureau and compared with other baseline models. The experimental results demonstrate that our model outperforms the compared baseline methods, proving the superiority and competitiveness of our model.

Data-driven Reactive Power Optimization for Distribution Networks Using Capsule Networks

The construction of advanced metering infrastructure and the rapid evolution of artificial intelligence bring opportunities to quickly searching for the optimal dispatching strategy for reactive power optimization. This can be realized by mining existing prior knowledge and massive data without explicitly constructing physical models. Therefore, a novel datadriven approach is proposed for reactive power optimization of distribution networks using capsule networks(CapsNet). The convolutional layers with strong feature extraction ability are used to project the power loads to the feature space to realize the automatic extraction of key features. Furthermore, the complex relationship between input features and dispatching strategies is captured accurately by capsule layers. The back propagation algorithm is utilized to complete the training process of the CapsNet. Case studies show that the accuracy and robustness of the CapsNet are better than those of popular baselines(e.g.,convolutional neural network, multi-layer perceptron, and casebased reasoning). Besides, the computing time is much lower than that of traditional heuristic methods such as genetic algorithm, which can meet the real-time demand of reactive power optimization in distribution networks.

Synchronized perturbation elimination and DOA estimation via signal selection mechanism and parallel deep capsule networks in multipath environment

State-of-the-art model-driven Direction-Of-Arrival(DOA)estimation methods for multipath signals face great challenges in practical application because of the dependence on the precise multipath model.In this paper,we introduce a framework,based on deep learning,for synchronizing perturbation auto-elimination with effective DOA estimation in multipath environment.Firstly,a signal selection mechanism is introduced to roughly locate specific signals to spatial subregion via frequency domain filters and compressive sensing-based method.Then,we set the mean of the correlation matrix’s row vectors as the input feature to construct the spatial spectrum by the corresponding single network within the parallel deep capsule networks.The proposed method enhances the generalization capability to untrained scenarios and the adaptability to non-ideal conditions,e.g.,lower SNRs,smaller snapshots,unknown reflection coefficients and perturbational steering vectors,which make up for the defects of the previous model-driven methods.Simulations are carried out to demonstrate the superiority of the proposed method.

Music Auto-Tagging with Capsule Network

In recent years,convolutional neural networks(CNNs)become popular approaches used in music information retrieval(MIR)tasks,such as mood recognition,music auto-tagging and so on.Since CNNs are able to extract the local features effectively,previous attempts show great performance on music auto-tagging.However,CNNs is not able to capture the spatial features and the relationship between low-level features are neglected.Motivated by this problem,a hybrid architecture is proposed based on Capsule Network,which is capable to extract spatial features with the routing-by-agreement mechanism.The proposed model was applied in music auto-tagging.The results show that it achieves promising results of the ROC-AUC score of 90.67%.

E3GCAPS: Efficient EEG-Based Multi-Capsule Framework with Dynamic Attention for Cross-Subject Cognitive State Detection

Cognitive state detection using electroencephalogram(EEG)signals for various tasks has attracted significant research attention.However,it is difficult to further improve the performance of crosssubject cognitive state detection.Further,most of the existing deep learning models will degrade significantly when limited training samples are given,and the feature hierarchical relationships are ignored.To address the above challenges,we propose an efficient interpretation model based on multiple capsule networks for cross-subject EEG cognitive state detection,termed as Efficient EEG-based Multi-Capsule Framework(E3GCAPS).Specifically,we use a selfexpression module to capture the potential connections between samples,which is beneficial to alleviate the sensitivity of outliers that are caused by the individual differences of cross-subject EEG.In addition,considering the strong correlation between cognitive states and brain function connection mode,the dynamic subcapsule-based spatial attention mechanism is introduced to explore the spatial relationship of multi-channel 1D EEG data,in which multichannel 1D data greatly improving the training efficiency while preserving the model performance.The effectiveness of the E3GCAPS is validated on the Fatigue-Awake EEG Dataset(FAAD)and the SJTU Emotion EEG Dataset(SEED).Experimental results show E3GCAPS can achieve remarkable results on the EEG-based cross-subject cognitive state detection under different tasks.

TibetanGoTinyNet:a lightweight U-Net style network for zero learning of Tibetan Go

The game of Tibetan Go faces the scarcity of expert knowledge and research literature.Therefore,we study the zero learning model of Tibetan Go under limited computing power resources and propose a novel scaleinvariant U-Net style two-headed output lightweight network TibetanGoTinyNet.The lightweight convolutional neural networks and capsule structure are applied to the encoder and decoder of TibetanGoTinyNet to reduce computational burden and achieve better feature extraction results.Several autonomous self-attention mechanisms are integrated into TibetanGoTinyNet to capture the Tibetan Go board’s spatial and global information and select important channels.The training data are generated entirely from self-play games.TibetanGoTinyNet achieves 62%–78%winning rate against other four U-Net style models including Res-UNet,Res-UNet Attention,Ghost-UNet,and Ghost Capsule-UNet.It also achieves 75%winning rate in the ablation experiments on the attention mechanism with embedded positional information.The model saves about 33%of the training time with 45%–50%winning rate for different Monte–Carlo tree search(MCTS)simulation counts when migrated from 9×9 to 11×11 boards.Code for our model is available at https://github.com/paulzyy/TibetanGoTinyNet.

User Station Security Protection Method Based on Random Domain Name Detection and Active Defense

The power monitoring system is the most important production management system in the power industry. As an important part of the power monitoring system, the user station that lacks grid binding will become an important target of network attacks. In order to perceive the network attack events on the user station side in time, a method combining real-time detection and active defense of random domain names on the user station side was proposed. Capsule network (CapsNet) combined with long short-term memory network (LSTM) was used to classify the domain names extracted from the traffic data. When a random domain name is detected, it sent instructions to routers and switched to update their security policies through the remote terminal protocol (Telnet), or shut down the service interfaces of routers and switched to block network attacks. The experimental results showed that the use of CapsNet combined with LSTM classification algorithm can achieve 99.16% accuracy and 98% recall rate in random domain name detection. Through the Telnet protocol, routers and switches can be linked to make active defense without interrupting services.

Automatic Extraction of Medical Latent Variables from ECG Signals Utilizing a Mutual Information-Based Technique and Capsular Neural Networks for Arrhythmia Detection

From a medical perspective,the 12 leads of the heart in an electrocardiogram(ECG)signal have functional dependencies with each other.Therefore,all these leads report different aspects of an arrhythmia.Their differences lie in the level of highlighting and displaying information about that arrhythmia.For example,although all leads show traces of atrial excitation,this function is more evident in lead II than in any other lead.In this article,a new model was proposed using ECG functional and structural dependencies between heart leads.In the prescreening stage,the ECG signals are segmented from the QRS point so that further analyzes can be performed on these segments in a more detailed manner.The mutual information indices were used to assess the relationship between leads.In order to calculate mutual information,the correlation between the 12 ECG leads has been calculated.The output of this step is a matrix containing all mutual information.Furthermore,to calculate the structural information of ECG signals,a capsule neural network was implemented to aid physicians in the automatic classification of cardiac arrhythmias.The architecture of this capsule neural network has been modified to perform the classification task.In the experimental results section,the proposed model was used to classify arrhythmias in ECG signals from the Chapman dataset.Numerical evaluations showed that this model has a precision of 97.02%,recall of 96.13%,F1-score of 96.57%and accuracy of 97.38%,indicating acceptable performance compared to other state-of-the-art methods.The proposed method shows an average accuracy of 2%superiority over similar works.

Improved Archimedes Optimization Algorithm with Deep Learning Empowered Fall Detection System

Human fall detection(FD)acts as an important part in creating sensor based alarm system,enabling physical therapists to minimize the effect of fall events and save human lives.Generally,elderly people suffer from several diseases,and fall action is a common situation which can occur at any time.In this view,this paper presents an Improved Archimedes Optimization Algorithm with Deep Learning Empowered Fall Detection(IAOA-DLFD)model to identify the fall/non-fall events.The proposed IAOA-DLFD technique comprises different levels of pre-processing to improve the input image quality.Besides,the IAOA with Capsule Network based feature extractor is derived to produce an optimal set of feature vectors.In addition,the IAOA uses to significantly boost the overall FD performance by optimal choice of CapsNet hyperparameters.Lastly,radial basis function(RBF)network is applied for determining the proper class labels of the test images.To showcase the enhanced performance of the IAOA-DLFD technique,a wide range of experiments are executed and the outcomes stated the enhanced detection outcome of the IAOA-DLFD approach over the recent methods with the accuracy of 0.997.

A Semantic Supervision Method for Abstractive Summarization

In recent years,many text summarization models based on pretraining methods have achieved very good results.However,in these text summarization models,semantic deviations are easy to occur between the original input representation and the representation that passed multi-layer encoder,which may result in inconsistencies between the generated summary and the source text content.The Bidirectional Encoder Representations from Transformers(BERT)improves the performance of many tasks in Natural Language Processing(NLP).Although BERT has a strong capability to encode context,it lacks the fine-grained semantic representation.To solve these two problems,we proposed a semantic supervision method based on Capsule Network.Firstly,we extracted the fine-grained semantic representation of the input and encoded result in BERT by Capsule Network.Secondly,we used the fine-grained semantic representation of the input to supervise the fine-grained semantic representation of the encoded result.Then we evaluated our model on a popular Chinese social media dataset(LCSTS),and the result showed that our model achieved higher ROUGE scores(including R-1,R-2),and our model outperformed baseline systems.Finally,we conducted a comparative study on the stability of the model,and the experimental results showed that our model was more stable.

IoT & AI Enabled Three-Phase Secure and Non-Invasive COVID 19 Diagnosis System

Corona is a viral disease that has taken the form of an epidemic and is causing havoc worldwide after its first appearance in the Wuhan state of China in December 2019.Due to the similarity in initial symptoms with viral fever,it is challenging to identify this virus initially.Non-detection of this virus at the early stage results in the death of the patient.Developing and densely populated countries face a scarcity of resources like hospitals,ventilators,oxygen,and healthcare workers.Technologies like the Internet of Things(IoT)and artificial intelligence can play a vital role in diagnosing the COVID-19 virus at an early stage.To minimize the spread of the pandemic,IoT-enabled devices can be used to collect patient’s data remotely in a secure manner.Collected data can be analyzed through a deep learning model to detect the presence of the COVID-19 virus.In this work,the authors have proposed a three-phase model to diagnose covid-19 by incorporating a chatbot,IoT,and deep learning technology.In phase one,an artificially assisted chatbot can guide an individual by asking about some common symptoms.In case of detection of even a single sign,the second phase of diagnosis can be considered,consisting of using a thermal scanner and pulse oximeter.In case of high temperature and low oxygen saturation levels,the third phase of diagnosis will be recommended,where chest radiography images can be analyzed through an AI-based model to diagnose the presence of the COVID-19 virus in the human body.The proposed model reduces human intervention through chatbot-based initial screening,sensor-based IoT devices,and deep learning-based X-ray analysis.It also helps in reducing the mortality rate by detecting the presence of the COVID-19 virus at an early stage.