Multimodal Sentiment Analysis Using BiGRU and Attention-Based Hybrid Fusion Strategy

Recently,multimodal sentiment analysis has increasingly attracted attention with the popularity of complementary data streams,which has great potential to surpass unimodal sentiment analysis.One challenge of multimodal sentiment analysis is how to design an efficient multimodal feature fusion strategy.Unfortunately,existing work always considers feature-level fusion or decision-level fusion,and few research works focus on hybrid fusion strategies that contain feature-level fusion and decision-level fusion.To improve the performance of multimodal sentiment analysis,we present a novel multimodal sentiment analysis model using BiGRU and attention-based hybrid fusion strategy(BAHFS).Firstly,we apply BiGRU to learn the unimodal features of text,audio and video.Then we fuse the unimodal features into bimodal features using the bimodal attention fusion module.Next,BAHFS feeds the unimodal features and bimodal features into the trimodal attention fusion module and the trimodal concatenation fusion module simultaneously to get two sets of trimodal features.Finally,BAHFS makes a classification with the two sets of trimodal features respectively and gets the final analysis results with decision-level fusion.Based on the CMU-MOSI and CMU-MOSEI datasets,extensive experiments have been carried out to verify BAHFS’s superiority.

Multi-Sensor Image Fusion: A Survey of the State of the Art

Image fusion has been developing into an important area of research. In remote sensing, the use of the same image sensor in different working modes, or different image sensors, can provide reinforcing or complementary information. Therefore, it is highly valuable to fuse outputs from multiple sensors (or the same sensor in different working modes) to improve the overall performance of the remote images, which are very useful for human visual perception and image processing task. Accordingly, in this paper, we first provide a comprehensive survey of the state of the art of multi-sensor image fusion methods in terms of three aspects: pixel-level fusion, feature-level fusion and decision-level fusion. An overview of existing fusion strategies is then introduced, after which the existing fusion quality measures are summarized. Finally, this review analyzes the development trends in fusion algorithms that may attract researchers to further explore the research in this field.

YOLOv2PD:An Efficient Pedestrian Detection Algorithm Using Improved YOLOv2 Model

Real-time pedestrian detection is an important task for unmanned driving systems and video surveillance.The existing pedestrian detection methods often work at low speed and also fail to detect smaller and densely distributed pedestrians by losing some of their detection accuracy in such cases.Therefore,the proposed algorithm YOLOv2(“YOU ONLY LOOK ONCE Version 2”)-based pedestrian detection(referred to as YOLOv2PD)would be more suitable for detecting smaller and densely distributed pedestrians in real-time complex road scenes.The proposed YOLOv2PD algorithm adopts a Multi-layer Feature Fusion(MLFF)strategy,which helps to improve the model’s feature extraction ability.In addition,one repeated convolution layer is removed from the final layer,which in turn reduces the computational complexity without losing any detection accuracy.The proposed algorithm applies the K-means clustering method on the Pascal Voc-2007+2012 pedestrian dataset before training to find the optimal anchor boxes.Both the proposed network structure and the loss function are improved to make the model more accurate and faster while detecting smaller pedestrians.Experimental results show that,at 544×544 image resolution,the proposed model achieves 80.7%average precision(AP),which is 2.1%higher than the YOLOv2 Model on the Pascal Voc-2007+2012 pedestrian dataset.Besides,based on the experimental results,the proposed model YOLOv2PD achieves a good trade-off balance between detection accuracy and real-time speed when evaluated on INRIA and Caltech test pedestrian datasets and achieves state-of-the-art detection results.

Fusion Model for Tentative Diagnosis Inference Based on Clinical Narratives

In general,physicians make a preliminary diagnosis based on patients’admission narratives and admission conditions,largely depending on their experiences and professional knowledge.An automatic and accurate tentative diagnosis based on clinical narratives would be of great importance to physicians,particularly in the shortage of medical resources.Despite its great value,little work has been conducted on this diagnosis method.Thus,in this study,we propose a fusion model that integrates the semantic and symptom features contained in the clinical text.The semantic features of the input text are initially captured by an attention-based Bidirectional Long Short-Term Memory(BiLSTM)network.The symptom concepts,recognized from the input text,are then vectorized by using the term frequency-inverse document frequency method based on the relations between symptoms and diseases.Finally,two fusion strategies are utilized to recommend the most potential candidate for the international classification of diseases code.Model training and evaluation are performed on a public clinical dataset.The results show that both fusion strategies achieved a promising performance,in which the best performance obtained a top-3 accuracy of 0.7412.

A Novel Fusion Forecast Model for Hail Weather in Plateau Areas Based on Machine Learning

In order to improve the accuracy of hail forecasting for mountainous and plateau areas in China,this study presents a novel fusion forecast model based on machine learning techniques.Specifically,known mechanisms of hail formation and two newly proposed elevation features calculated from radar data,sounding data,automatic station data,and terrain data,are firstly combined,from which a hail/short-duration heavy rainfall(SDHR)classification model based on the random forest(RF)algorithm is built up.Then,we construct a hail/SDHR probability identification(PI)model based on the Bayesian minimum error decision and principal component analysis methods.Finally,an"and"fusion strategy for coupling the RF and PI models is proposed.In addition to the mechanism features,the new elevation features improve the models’performance significantly.Experimental results show that the fusion strategy is particularly notable for reducing the number of false alarms on the premise of ensuring the hit rate.A comparison with two classical hail indexes shows that our proposed algorithm has a higher forecasting accuracy for hail in mountainous and plateau areas.All 19 hail cases used for testing could be identified,and our algorithm is able to provide an early warning for 89.5%(17 cases)of hail cases,among which 52.6%(10 cases)receive an early warning of more than 42 minutes in advance.The PI model sheds new light on using Bayesian classification approaches for highdimensional solutions.

Ensemble deep transfer learning driven by multisensor signals for the fault diagnosis of bevel-gear cross-operation conditions

The existing intelligent fault diagnosis techniques of bevel gear focus on single-sensor signal analysis under the steady operation condition.In this study,a new method is proposed based on ensemble deep transfer learning and multisensor signals to enhance the fault diagnosis adaptability and reliability of bevel gear under various operation conditions.First,a novel stacked autoencoder(NSAE)is constructed using a denoising autoencoder,batch normalization,and the Swish activation function.Second,a series of source-domain NSAEs with multisensor vibration signals is pretrained.Third,the good model parameters provided by the source-domain NSAEs are transferred to initialize the corresponding target-domain NSAEs.Finally,a modified voting fusion strategy is designed to obtain a comprehensive result.The multisensor signals collected under the different operation conditions of bevel gear are used to verify the proposed method.The comparison results show that the proposed method can diagnose different faults in an accurate and stable manner using only one target-domain sample,thereby outperforming the existing methods.

An efficient GPU-based parallel tabu search algorithm for hardware/software co-design

Hardware/software partitioning is an essential step in hardware/software co-design.For large size problems,it is difficult to consider both solution quality and time.This paper presents an efficient GPU-based parallel tabu search algorithm(GPTS)for HW/SW partitioning.A single GPU kernel of compacting neighborhood is proposed to reduce the amount of GPU global memory accesses theoretically.A kernel fusion strategy is further proposed to reduce the amount of GPU global memory accesses of GPTS.To further minimize the transfer overhead of GPTS between CPU and GPU,an optimized transfer strategy for GPU-based tabu evaluation is proposed,which considers that all the candidates do not satisfy the given constraint.Experiments show that GPTS outperforms state-of-the-art work of tabu search and is competitive with other methods for HW/SW partitioning.The proposed parallelization is significant when considering the ordinary GPU platform.

A Novel Approach of Hybrid Genetic Optimization with Elite Ant Colony:Application in the Modulation Recognition

A novel approach （HGO-EAC） for hybrid genetic op-timization （GO） with elite ant colony （EAC） is proposed for the automatic modulation recognition of communication signals,through which we improve the basic ant colony algorithms by referencing elite strategy and present a new fusion strategy for genetic optimization and elite ant colony. This approach is used to train the neural networks as the classifier for modulation. Simula-tion results indicate good performance on an additive white Gaus-sian noise （AWGN） channel,with recognition rate reaching to 70% especially for CW even at signal-to-noise ratios as low as 5 dB. This approach can achieve a high recognition rate for the typical modulations such as CW,4ASK,4FSK,BPSK,and QAM16. Test result shows that it has better performance than BP algorithm and basic ant colony algorithms by achieving faster training and stronger robustness.