Multi-View & Transfer Learning for Epilepsy Recognition Based on EEG Signals

Epilepsy is a central nervous system disorder in which brain activity becomes abnormal.Electroencephalogram(EEG)signals,as recordings of brain activity,have been widely used for epilepsy recognition.To study epilep-tic EEG signals and develop artificial intelligence(AI)-assist recognition,a multi-view transfer learning(MVTL-LSR)algorithm based on least squares regression is proposed in this study.Compared with most existing multi-view transfer learning algorithms,MVTL-LSR has two merits:(1)Since traditional transfer learning algorithms leverage knowledge from different sources,which poses a significant risk to data privacy.Therefore,we develop a knowledge transfer mechanism that can protect the security of source domain data while guaranteeing performance.(2)When utilizing multi-view data,we embed view weighting and manifold regularization into the transfer framework to measure the views’strengths and weaknesses and improve generalization ability.In the experimental studies,12 different simulated multi-view&transfer scenarios are constructed from epileptic EEG signals licensed and provided by the Uni-versity of Bonn,Germany.Extensive experimental results show that MVTL-LSR outperforms baselines.The source code will be available on https://github.com/didid5/MVTL-LSR.

Contrastive Consistency and Attentive Complementarity for Deep Multi-View Subspace Clustering

Deep multi-view subspace clustering (DMVSC) based on self-expression has attracted increasing attention dueto its outstanding performance and nonlinear application. However, most existing methods neglect that viewprivatemeaningless information or noise may interfere with the learning of self-expression, which may lead to thedegeneration of clustering performance. In this paper, we propose a novel framework of Contrastive Consistencyand Attentive Complementarity (CCAC) for DMVsSC. CCAC aligns all the self-expressions of multiple viewsand fuses them based on their discrimination, so that it can effectively explore consistent and complementaryinformation for achieving precise clustering. Specifically, the view-specific self-expression is learned by a selfexpressionlayer embedded into the auto-encoder network for each view. To guarantee consistency across views andreduce the effect of view-private information or noise, we align all the view-specific self-expressions by contrastivelearning. The aligned self-expressions are assigned adaptive weights by channel attention mechanism according totheir discrimination. Then they are fused by convolution kernel to obtain consensus self-expression withmaximumcomplementarity ofmultiple views. Extensive experimental results on four benchmark datasets and one large-scaledataset of the CCAC method outperformother state-of-the-artmethods, demonstrating its clustering effectiveness.

Multi-view Feature Learning for the Over-penalty in Adversarial Domain Adaptation

Domain adaptation aims to transfer knowledge from the labeled source domain to an unlabeled target domain that follows a similar but different distribution.Recently,adversarial-based methods have achieved remarkable success due to the excellent performance of domain-invariant feature presentation learning.However,the adversarial methods learn the transferability at the expense of the discriminability in feature representation,leading to low generalization to the target domain.To this end,we propose a Multi-view Feature Learning method for the Over-penalty in Adversarial Domain Adaptation.Specifically,multi-view representation learning is proposed to enrich the discriminative information contained in domain-invariant feature representation,which will counter the over-penalty for discriminability in adversarial training.Besides,the class distribution in the intra-domain is proposed to replace that in the inter-domain to capture more discriminative information in the learning of transferrable features.Extensive experiments show that our method can improve the discriminability while maintaining transferability and exceeds the most advanced methods in the domain adaptation benchmark datasets.

Incremental Data Stream Classification with Adaptive Multi-Task Multi-View Learning

With the enhancement of data collection capabilities,massive streaming data have been accumulated in numerous application scenarios.Specifically,the issue of classifying data streams based on mobile sensors can be formalized as a multi-task multi-view learning problem with a specific task comprising multiple views with shared features collected from multiple sensors.Existing incremental learning methods are often single-task single-view,which cannot learn shared representations between relevant tasks and views.An adaptive multi-task multi-view incremental learning framework for data stream classification called MTMVIS is proposed to address the above challenges,utilizing the idea of multi-task multi-view learning.Specifically,the attention mechanism is first used to align different sensor data of different views.In addition,MTMVIS uses adaptive Fisher regularization from the perspective of multi-task multi-view learning to overcome catastrophic forgetting in incremental learning.Results reveal that the proposed framework outperforms state-of-the-art methods based on the experiments on two different datasets with other baselines.

Multi-view feature fusion for rolling bearing fault diagnosis using random forest and autoencoder

To improve the accuracy and robustness of rolling bearing fault diagnosis under complex conditions, a novel method based on multi-view feature fusion is proposed. Firstly, multi-view features from perspectives of the time domain, frequency domain and time-frequency domain are extracted through the Fourier transform, Hilbert transform and empirical mode decomposition (EMD).Then, the random forest model (RF) is applied to select features which are highly correlated with the bearing operating state. Subsequently, the selected features are fused via the autoencoder (AE) to further reduce the redundancy. Finally, the effectiveness of the fused features is evaluated by the support vector machine (SVM). The experimental results indicate that the proposed method based on the multi-view feature fusion can effectively reflect the difference in the state of the rolling bearing, and improve the accuracy of fault diagnosis.

Feature Fusion Multi-View Hashing Based on Random Kernel Canonical Correlation Analysis

Hashing technology has the advantages of reducing data storage and improving the efficiency of the learning system,making it more and more widely used in image retrieval.Multi-view data describes image information more comprehensively than traditional methods using a single-view.How to use hashing to combine multi-view data for image retrieval is still a challenge.In this paper,a multi-view fusion hashing method based on RKCCA(Random Kernel Canonical Correlation Analysis)is proposed.In order to describe image content more accurately,we use deep learning dense convolutional network feature DenseNet to construct multi-view by combining GIST feature or BoW_SIFT(Bag-of-Words model+SIFT feature)feature.This algorithm uses RKCCA method to fuse multi-view features to construct association features and apply them to image retrieval.The algorithm generates binary hash code with minimal distortion error by designing quantization regularization terms.A large number of experiments on benchmark datasets show that this method is superior to other multi-view hashing methods.

Transfer Learning via Multi-View Principal Component Analysis

Transfer learning aims at leveraging the knowledge in labeled source domains to predict the unlabeled data in a target domain, where the distributions are diiTerent in domains. Among various methods for transfer learning, one kind of Mgorithms focus on the correspondence between bridge features and all the other specific features from different domains, and later conduct transfer learning via the single-view correspondence. However, the single-view correspondence may prevent these algorithms from further improvement due to the problem of incorrect correlation discovery. To tackle this problem, we propose a new method for transfer learning in a multi-view correspondence perspective, which is called MultiView Principal Component Analysis （MVPCA） approach. MVPCA discovers the correspondence between bridge features representative across all domains and specific features from different domains respectively, and conducts the transfer learning by dimensionality reduction in a multi-view way, which can better depict the knowledge transfer. Experiments show that MVPCA can significantly reduce the cross domain prediction error of a baseline non-transfer method. With multi-view correspondence information incorporated to the single-view transfer learning method, MVPCA can further improve the performance of one state-of-the-art single-view method.

Learning multi-kernel multi-view canonical correlations for image recognition

In this paper, we propose a multi-kernel multi-view canonical correlations(M2CCs) framework for subspace learning. In the proposed framework,the input data of each original view are mapped into multiple higher dimensional feature spaces by multiple nonlinear mappings determined by different kernels. This makes M2 CC can discover multiple kinds of useful information of each original view in the feature spaces. With the framework, we further provide a specific multi-view feature learning method based on direct summation kernel strategy and its regularized version. The experimental results in visual recognition tasks demonstrate the effectiveness and robustness of the proposed method.

Label-noise robust classification with multi-view learning

Label noise is often contained in the training data due to various human factors or measurement errors,which significantly causes a negative effect on classifiers.Despite many previous methods that have been proposed to learn robust classifiers,they are mainly based on the single-view feature.On the other hand,although existing multi-view classification methods benefit from the more comprehensive information,they rarely consider label noise.In this paper,we propose a novel label-noise robust classification model with multi-view learning to overcome these limitations.In the proposed model,not only the classifier learning but also the label-noise removal can benefit from the multi-view information.Specifically,we relax the label matrix of the basic multi-view least squares regression model,and develop a nonlinear transformation with a natural probabilistic approximation in the process of labels,which is conveniently optimized and beneficial to improve the discriminative ability of classifiers.Moreover,we preserve the intrinsic manifold structure of multi-view data on the relaxed label matrix,facilitating the process of label relaxation.For optimizing the proposed model with the nonlinear transformation,we derive a lemma about the partial derivation of the softmax related function,and develop an efficient alternating algorithm.Experimental evaluations on six real-world datasets confirm the advantages of the proposed method,compared to the related state-of-the-art methods.

View-invariant human action recognition via robust locally adaptive multi-view learning

Human action recognition is currently one of the most active research areas in computer vision. It has been widely used in many applications, such as intelligent surveillance, perceptual interface, and content-based video retrieval. However, some extrinsic factors are barriers for the development of action recognition; e.g., human actions may be observed from arbitrary camera viewpoints in realistic scene. Thus, view-invariant analysis becomes important for action recognition algorithms, and a number of researchers have paid much attention to this issue. In this paper, we present a multi-view learning approach to recognize human actions from different views. As most existing multi-view learning algorithms often suffer from the problem of lacking data adaptiveness in the nearest neighborhood graph construction procedure, a robust locally adaptive multi-view learning algorithm based on learning multiple local L 1-graphs is proposed. Moreover, an efficient iterative optimization method is proposed to solve the proposed objective function. Experiments on three public view-invariant action recognition datasets, i.e., ViHASi, IXMAS, and WVU, demonstrate data adaptiveness, effectiveness, and efficiency of our algorithm. More importantly, when the feature dimension is correctly selected （i.e., 〉60）, the proposed algorithm stably outperforms state-of-the-art counterparts and obtains about 6% improvement in recognition accuracy on the three datasets.

Hypergraph regularized multi-view subspace clustering with dual tensor log-determinant

The existing multi-view subspace clustering algorithms based on tensor singular value decomposition(t-SVD)predominantly utilize tensor nuclear norm to explore the intra view correlation between views of the same samples,while neglecting the correlation among the samples within different views.Moreover,the tensor nuclear norm is not fully considered as a convex approximation of the tensor rank function.Treating different singular values equally may result in suboptimal tensor representation.A hypergraph regularized multi-view subspace clustering algorithm with dual tensor log-determinant(HRMSC-DTL)was proposed.The algorithm used subspace learning in each view to learn a specific set of affinity matrices,and introduced a non-convex tensor log-determinant function to replace the tensor nuclear norm to better improve global low-rankness.It also introduced hyper-Laplacian regularization to preserve the local geometric structure embedded in the high-dimensional space.Furthermore,it rotated the original tensor and incorporated a dual tensor mechanism to fully exploit the intra view correlation of the original tensor and the inter view correlation of the rotated tensor.At the same time,an alternating direction of multipliers method(ADMM)was also designed to solve non-convex optimization model.Experimental evaluations on seven widely used datasets,along with comparisons to several state-of-the-art algorithms,demonstrated the superiority and effectiveness of the HRMSC-DTL algorithm in terms of clustering performance.

Overview of 3D Human Pose Estimation

3D human pose estimation is a major focus area in the field of computer vision,which plays an important role in practical applications.This article summarizes the framework and research progress related to the estimation of monocular RGB images and videos.An overall perspective ofmethods integrated with deep learning is introduced.Novel image-based and video-based inputs are proposed as the analysis framework.From this viewpoint,common problems are discussed.The diversity of human postures usually leads to problems such as occlusion and ambiguity,and the lack of training datasets often results in poor generalization ability of the model.Regression methods are crucial for solving such problems.Considering image-based input,the multi-view method is commonly used to solve occlusion problems.Here,the multi-view method is analyzed comprehensively.By referring to video-based input,the human prior knowledge of restricted motion is used to predict human postures.In addition,structural constraints are widely used as prior knowledge.Furthermore,weakly supervised learningmethods are studied and discussed for these two types of inputs to improve the model generalization ability.The problem of insufficient training datasets must also be considered,especially because 3D datasets are usually biased and limited.Finally,emerging and popular datasets and evaluation indicators are discussed.The characteristics of the datasets and the relationships of the indicators are explained and highlighted.Thus,this article can be useful and instructive for researchers who are lacking in experience and find this field confusing.In addition,by providing an overview of 3D human pose estimation,this article sorts and refines recent studies on 3D human pose estimation.It describes kernel problems and common useful methods,and discusses the scope for further research.

快速多视角特权协同随机向量函数连接网络

现实情况中通常会针对同一对象从不同途径或层面获得特征数据,称这样获得的数据为多视角数据。对于多视角数据的挖掘利用具有研究价值,与传统的单视角学习相比表现出一定优势。多视角学习(MVL)中一个重要的问题是如何在满足视角间互补情况下同时保持视角之间一致性。为解决上述问题,基于多视角学习和特权信息学习(LUPI)概念,以随机向量函数连接网络(RVFL)为基础,提出了一种快速多视角特权协同随机向量函数连接网络(FMPRVFL)来有效地解决多视角分类任务。该方法的基本思想是在平均情况下相互利用冗余视角的附加信息作为特权信息监督当前视角的分类。在此基础上设计的FMPRVFL的目标函数可以利用解析解对目标函数进行优化,从而使FMPRVFL训练速度更快。理论分析表明,与经典的多视角学习方法相比,FMPRVFL可以提供额外的泛化能力。在64个数据集上的实验结果表明,FMPRVFL在平均测试精度和运行时间上都优于比较方法。

Co-metric： a metric learning algorithm for data with multiple views

We address the problem of metric learning for multi-view data. Many metric learning algorithms have been proposed, most of them focus just on single view circumstances, and only a few deal with multi-view data. In this paper, motivated by the co-training framework, we propose an algorithm-independent framework, named co-metric, to learn Mahalanobis metrics in multi-view settings. In its implementation, an off-the-shelf single-view metric learning algorithm is used to learn metrics in individual views of a few labeled examples. Then the most confidently-labeled examples chosen from the unlabeled set are used to guide the metric learning in the next loop. This procedure is repeated until some stop criteria are met. The framework can accommodate most existing metric learning algorithms whether types-of- side-information or example-labels are used. In addition it can naturally deal with semi-supervised circumstances under more than two views. Our comparative experiments demon- strate its competiveness and effectiveness.

核化的多视角特权协同随机矢量功能链接网络及其增量学习方法

在许多实际应用场景中,可以从不同层次、不同角度获取相同对象的特征数据,如何有效地利用获取的多视角数据是一个值得研究的问题.和传统的单视角学习相比,多视角学习在多源数据的应用中显示了一定的优势.多角度学习(Multi-View Learning,MVL)面临的一个重要问题是在满足不同视角互补性的前提下如何保持视角之间的一致性.针对以上问题,提出一种新的多视角特权协同核化随机向量功能链接网络(KMPRVFL)来有效地解决多视角分类问题,其基本思想是将冗余视角的额外信息与平均视角上的特权信息相结合来监督当前视角的分类任务,将多视角数据用核化后加权线性组合成综合第二视角.同时,还设计了一种增量学习方法,可以有效地减少计算量.在真实数据集上的实验结果表明,和传统的多视角学习方法相比,KMPRVFL的能力更强,其平均测试精度要优于对比算法.

Semi-supervised multi-view clustering with dual hypergraph regularized partially shared non-negative matrix factorization

Real-world data can often be represented in multiple forms and views,and analyzing data from different perspectives allows for more comprehensive learning of the data,resulting in better data clustering results.Non-negative matrix factorization(NMF)is used to solve the clustering problem to extract uniform discriminative low-dimensional features from multi-view data.Many clustering methods based on graph regularization have been proposed and proven to be effective,but ordinary graphs only consider pairwise relationships between samples.In order to learn the higher-order relationships that exist in the sample manifold and feature manifold of multi-view data,we propose a new semi-supervised multi-view clustering method called dual hypergraph regularized partially shared non-negative matrix factorization(DHPS-NMF).The complex manifold structure of samples and features is learned by constructing samples and feature hypergraphs.To improve the discrimination power of the obtained lowdimensional features,semi-supervised regression terms are incorporated into the model to effectively use the label information when capturing the complex manifold structure of the data.Ultimately,we conduct experiments on six real data sets and the results show that our algorithm achieves encouraging results in comparison with some methods.

Multi-view Clustering: A Survey

In the big data era, the data are generated from different sources or observed from different views. These data are referred to as multi-view data. Unleashing the power of knowledge in multi-view data is very important in big data mining and analysis. This calls for advanced techniques that consider the diversity of different views,while fusing these data. Multi-view Clustering(MvC) has attracted increasing attention in recent years by aiming to exploit complementary and consensus information across multiple views. This paper summarizes a large number of multi-view clustering algorithms, provides a taxonomy according to the mechanisms and principles involved, and classifies these algorithms into five categories, namely, co-training style algorithms, multi-kernel learning, multiview graph clustering, multi-view subspace clustering, and multi-task multi-view clustering. Therein, multi-view graph clustering is further categorized as graph-based, network-based, and spectral-based methods. Multi-view subspace clustering is further divided into subspace learning-based, and non-negative matrix factorization-based methods. This paper does not only introduce the mechanisms for each category of methods, but also gives a few examples for how these techniques are used. In addition, it lists some publically available multi-view datasets.Overall, this paper serves as an introductory text and survey for multi-view clustering.

Multiple hypergraph ranking for video concept detection

This paper tackles the problem of video concept detection using the multi-modality fusion method. Motivated by multi-view learning algorithms, multi-modality features of videos can be represented by multiple graphs. And the graph-based semi-supervised learning methods can be extended to multiple graphs to predict the semantic labels for unlabeled video data. However, traditional graphs represent only homogeneous pairwise linking relations, and therefore the high-order correlations inherent in videos, such as high-order visual similarities, are ignored. In this paper we represent heterogeneous features by multiple hypergraphs and then the high-order correlated samples can be associated with hyperedges. Furthermore, the multi-hypergraph ranking (MHR) algorithm is proposed by defining Markov random walk on each hypergraph and then forming the mixture Markov chains so as to perform transductive learning in multiple hypergraphs. In experiments on the TRECVID dataset, a triple-hypergraph consisting of visual, textual features and multiple labeled tags is constructed to predict concept labels for unlabeled video shots by the MHR framework. Experimental results show that our approach is effective.

TranSR-Ne RF:Super-resolution neural radiance field for reconstruction and rendering of weak and repetitive texture of aviation damaged functional surface

In order to reconstruct and render the weak and repetitive texture of the damaged functional surface of aviation,an improved neural radiance field,named TranSR-NeRF,is proposed.In this paper,a data acquisition system was designed and built.The acquired images generated initial point clouds through TransMVSNet.Meanwhile,after extracting features from the images through the improved SE-ConvNeXt network,the extracted features were aligned and fused with the initial point cloud to generate high-quality neural point cloud.After ray-tracing and sampling of the neural point cloud,the ResMLP neural network designed in this paper was used to regress the volume density and radiance under a given viewing angle,which introduced spatial coordinate and relative positional encoding.The reconstruction and rendering of arbitrary-scale super-resolution of damaged functional surface is realized.In this paper,the influence of illumination conditions and background environment on the model performance is also studied through experiments,and the comparison and ablation experiments for the improved methods proposed in this paper is conducted.The experimental results show that the improved model has good effect.Finally,the application experiment of object detection task is carried out,and the experimental results show that the model has good practicability.