A Robust Framework for Multimodal Sentiment Analysis with Noisy Labels Generated from Distributed Data Annotation

Multimodal sentiment analysis utilizes multimodal data such as text,facial expressions and voice to detect people’s attitudes.With the advent of distributed data collection and annotation,we can easily obtain and share such multimodal data.However,due to professional discrepancies among annotators and lax quality control,noisy labels might be introduced.Recent research suggests that deep neural networks(DNNs)will overfit noisy labels,leading to the poor performance of the DNNs.To address this challenging problem,we present a Multimodal Robust Meta Learning framework(MRML)for multimodal sentiment analysis to resist noisy labels and correlate distinct modalities simultaneously.Specifically,we propose a two-layer fusion net to deeply fuse different modalities and improve the quality of the multimodal data features for label correction and network training.Besides,a multiple meta-learner(label corrector)strategy is proposed to enhance the label correction approach and prevent models from overfitting to noisy labels.We conducted experiments on three popular multimodal datasets to verify the superiority of ourmethod by comparing it with four baselines.

Data fusing and joint training for learning with noisy labels

It is well known that deep learning depends on a large amount of clean data.Because of high annotation cost,various methods have been devoted to annotating the data automatically.However,a larger number of the noisy labels are generated in the datasets,which is a challenging problem.In this paper,we propose a new method for selecting training data accurately.Specifically,our approach fits a mixture model to the per-sample loss of the raw label and the predicted label,and the mixture model is utilized to dynamically divide the training set into a correctly labeled set,a correctly predicted set,and a wrong set.Then,a network is trained with these sets in the supervised learning manner.Due to the confirmation bias problem,we train the two networks alternately,and each network establishes the data division to teach the other network.When optimizing network parameters,the labels of the samples fuse respectively by the probabilities from the mixture model.Experiments on CIFAR-10,CIFAR-100 and Clothing1M demonstrate that this method is the same or superior to the state-of-the-art methods.

Injecting spectral indices to transferable convolutional neural network under imbalanced and noisy labels for Landsat image classification

Stable and continuous remote sensing land-cover mapping is important for agriculture,ecosystems,and land management.Convolutional neural networks(CNNs)are promising methods for achieving this goal.However,the large number of high-quality training samples required to train a CNN is difficult to acquire.In practice,imbalanced and noisy labels originating from existing land-cover maps can be used as alternatives.Experiments have shown that the inconsistency in the training samples has a significant impact on the performance of the CNN.To overcome this drawback,a method is proposed to inject highly consistent information into the network,to learn general and transferable features to alleviate the impact of imperfect training samples.Spectral indices are important features that can provide consistent information.These indices can be fused with CNN feature maps which utilize information entropy to choose the most appropriate CNN layer,to compensate for the inconsistency caused by the imbalanced,noisy labels.The proposed transferable CNN,tested with imbalanced and noisy labels for inter-regional Landsat time-series,not only is superior in terms of accuracy for land-cover mapping but also demonstrates excellent transferability between regions in both time series and cross-regional Landsat image classification.

A robust optimization method for label noisy datasets based on adaptive threshold: Adaptive-k

The use of all samples in the optimization process does not produce robust results in datasets with label noise.Because the gradients calculated according to the losses of the noisy samples cause the optimization process to go in the wrong direction.In this paper,we recommend using samples with loss less than a threshold determined during the optimization,instead of using all samples in the mini-batch.Our proposed method,Adaptive-k,aims to exclude label noise samples from the optimization process and make the process robust.On noisy datasets,we found that using a threshold-based approach,such as Adaptive-k,produces better results than using all samples or a fixed number of low-loss samples in the mini-batch.On the basis of our theoretical analysis and experimental results,we show that the Adaptive-k method is closest to the performance of the Oracle,in which noisy samples are entirely removed from the dataset.Adaptive-k is a simple but effective method.It does not require prior knowledge of the noise ratio of the dataset,does not require additional model training,and does not increase training time significantly.In the experiments,we also show that Adaptive-k is compatible with different optimizers such as SGD,SGDM,and Adam.The code for Adaptive-k is available at GitHub.

Semi-Supervised Noisy Label Learning for Chinese Clinical Named Entity Recognition

This paper describes our approach for the Chinese clinical named entity recognition(CNER) task organized by the 2020 China Conference on Knowledge Graph and Semantic Computing(CCKS) competition. In this task, we need to identify the entity boundary and category labels of six entities from Chinese electronic medical record(EMR). We constructed a hybrid system composed of a semi-supervised noisy label learning model based on adversarial training and a rule post-processing module. The core idea of the hybrid system is to reduce the impact of data noise by optimizing the model results. Besides, we used post-processing rules to correct three cases of redundant labeling, missing labeling, and wrong labeling in the model prediction results. Our method proposed in this paper achieved strict criteria of 0.9156 and relax criteria of 0.9660 on the final test set, ranking first.

Active label-denoising algorithm based on broad learning for annotation of machine health status

Deep learning has led to tremendous success in machine maintenance and fault diagnosis.However,this success is predicated on the correctly annotated datasets.Labels in large industrial datasets can be noisy and thus degrade the performance of fault diagnosis models.The emerging concept of broad learning shows the potential to address the label noise problem.Compared with existing deep learning algorithms,broad learning has a simple architecture and high training efficiency.An active label denoising algorithm based on broad learning(ALDBL)is proposed.First,ALDBL captures the embedded representation from the time-frequency features by a recurrent memory cell.Second,it augments wide features with a sparse autoencoder and projects the sparse features into an orthogonal space.A proposed corrector then iteratively changes the weights of source examples during the training and corrects the labels by using a label adaptation matrix.Finally,ALDBL finetunes the model parameters with actively sampled target data with reliable pseudo labels.The performance of ALDBL is validated with three benchmark datasets,including 30 label denoising tasks.Computational results demonstrate the effectiveness and advantages of the proposed algorithm over the other label denoising algorithms.