Two-Phase Genetic Algorithm Applied in the Optimization of Multi-Modal Function

This paper presents a two-phase genetic algorithm (TPGA) based on the multi- parent genetic algorithm (MPGA). Through analysis we find MPGA will lead the population' s evol vement to diversity or convergence according to the population size and the crossover size, so we make it run in different forms during the global and local optimization phases and then forms TPGA. The experiment results show that TPGA is very efficient for the optimization of low-dimension multi-modal functions, usually we can obtain all the global optimal solutions.

Multi-modal Gesture Recognition using Integrated Model of Motion, Audio and Video

Gesture recognition is used in many practical applications such as human-robot interaction, medical rehabilitation and sign language. With increasing motion sensor development, multiple data sources have become available, which leads to the rise of multi-modal gesture recognition. Since our previous approach to gesture recognition depends on a unimodal system, it is difficult to classify similar motion patterns. In order to solve this problem, a novel approach which integrates motion, audio and video models is proposed by using dataset captured by Kinect. The proposed system can recognize observed gestures by using three models. Recognition results of three models are integrated by using the proposed framework and the output becomes the final result. The motion and audio models are learned by using Hidden Markov Model. Random Forest which is the video classifier is used to learn the video model. In the experiments to test the performances of the proposed system, the motion and audio models most suitable for gesture recognition are chosen by varying feature vectors and learning methods. Additionally, the unimodal and multi-modal models are compared with respect to recognition accuracy. All the experiments are conducted on dataset provided by the competition organizer of MMGRC, which is a workshop for Multi-Modal Gesture Recognition Challenge. The comparison results show that the multi-modal model composed of three models scores the highest recognition rate. This improvement of recognition accuracy means that the complementary relationship among three models improves the accuracy of gesture recognition. The proposed system provides the application technology to understand human actions of daily life more precisely.

A Two-Level Subspace Evolutionary Algorithm for Solving Multi-Modal Function Optimization Problems

In this paper, a new algorithm for solving multi-modal function optimization problems-two-level subspace evolutionary algorithm is proposed. In the first level, the improved GT algorithm is used to do global recombination search so that the whole population can be separated into several niches according to the position of solutions; then, in the second level, the niche evolutionary strategy is used for local search in the subspaces gotten in the first level till solutions of the problem are found. The new algorithm has been tested on some hard problems and some good results are obtained.

Multi-Modal Domain Adaptation Variational Autoencoder for EEG-Based Emotion Recognition

Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.

Analysis of prompt fission neutron spectrum and multiplicity for ^(237)Np(n, f) in the frame of multi-modal Los Alamos model

The improved version of Los Alamos model with the multi-modal fission approach is used to analyse the prompt fission neutron spectrum and multiplicity for the neutron-induced fission of 237Np. The spectra of neutrons emitted from fragments for the three most dominant fission modes （standard Ⅰ, standard Ⅱ and superlong） are calculated separately and the total spectrum is synthesized. The multi-modal parameters contained in the spectrum model are determined on the basis of experimental data of fission fragment mass distributions. The calculated total prompt fission neutron spectrum and multiplicity are better agreement with the experimental data than those obtained from the conventional treatment of the Los Alamos model.

Calculation of Prompt Fission Neutron from ^(233)U(n, f) Reaction byMulti-Modal Los Alamos Model

An attempt is made to improve the evaluation of the prompt fission neutron emis- sion from 233U（n, f） reaction for incident neutron energies below 6 MeV. The multi-modal fission approach is applied to the improved version of Los Alamos model and the point by point model. The prompt fission neutron spectra and the prompt fission neutron as a function of fragment mass （usually named ＂sawtooth＂ data） v（A） are calculated independently for the three most dominant fission modes （standard I, standard II and superlong）, and the total spectra and v（A） are syn- thesized. The multi-modal parameters are determined on the basis of experimental data of fission fragment mass distributions. The present calculation results can describe the experimental data very well, and the proposed treatment is thus a useful tool for prompt fission neutron emission prediction.

Information perception and feedback mechanism and key techniques of multi-modality human-robot interaction for service robots

With the increasing of the elderly population and the growing hearth care cost, the role of service robots in aiding the disabled and the elderly is becoming important. Many researchers in the world have paid much attention to heaRthcare robots and rehabilitation robots. To get natural and harmonious communication between the user and a service robot, the information perception/feedback ability, and interaction ability for service robots become more important in many key issues.

Multi-modal human-machine interface of a telerobotic system for remote arc welding

In telerobotic system for remote welding, human-machine interface is one of the most important factor for enhancing capability and efficiency. This paper presents an architecture design of human-machine interface for welding telerobotic system： welding multi-modal human-machine interface. The human-machine interface integrated several control modes, which are namely shared control, teleteaching, supervisory control and local autonomous control. Space mouse, panoramic vision camera and graphics simulation system are also integrated into the human-machine interface for welding teleoperation. Finally, weld seam tracing and welding experiments of U-shape seam are performed by these control modes respectively. The results show that the system has better performance of human-machine interaction and complexity environment welding.

Method for Detecting Multi-Modal Vibration Characteristics of Landmines

The acoustic vibration characteristics of landmines are investigated by means of modal analysis. According to the mechanical structure of landmines, a certain number of points are marked on the landmine shell to analyze its multi-modal vibration characteristics, based on laser self-mixing interferometer and taking 69 plastic landmine as an example, the vibration detection experiment system is built to show the results of analytical method of multi-modal testing. The first and second order natural frequencies of the bricks are 38 HZ and 106 HZ, 112 HZ and 232 HZ for plastic landmines, and 74 HZ and 290 HZ for metal landmines. The first and second order natural frequencies of the bricks are far smaller than those of plastic landmines and metal landmines. This indicates that landmines show multi-modal vibration characteristics under external excitation, which are significantly different from those of bricks. The findings can be used for further research on acoustic landmines detection technology.

基于MD-BERT-LGBM的智能诊疗与预测模型设计

针对目前医疗资源紧张且难以实现精准预测病情发展的问题,文中提出了一种基于MD-BERT-LGBM的智能诊疗与预测模型。该模型引入了多模态高维度向量来表征信息,以此提高模型精确度。采用双向编码BERT替代传统的CNN算法来处理文本信息数据,同时在每个BERT模块内融合了KMP识别算法。通过归一化指数函数来预训练和校准BERT模型,用LGBM机器学习算法高效地处理表征多模态高维度向量数据。多组实验测试结果表明,与同类的CNN-MDRP模型相比,所提模型的诊疗与预测精确度分别提升了1.1%和4.0%,具有良好的可靠性。

The Construal of Multi-modal Metaphor in Public Service Advertisements between China and America

Based on the theory of Forceville’s multi-modal metaphor,this paper adopts qualitative and quantitative research methods to analyze 60 social safety ads both in China and America,trying to demonstrate the similarities and differences between the chosen social safety ads in using multi-modal metaphor and discussing the factors that caused these differences.

M^(2)LC-Net: A Multi-Modal Multi-Disease Long-Tailed Classification Network for Real Clinical Scenes

Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the number of ophthalmic diseases that can be classified is relatively small.Moreover,imbalanced data distribution of different ophthalmic diseases is not taken into consideration,which limits the application of deep learning techniques in realistic clinical scenes.In this paper,we propose a Multimodal Multi-disease Long-tailed Classification Network(M^(2)LC-Net)in response to the challenges mentioned above.M^(2)LC-Net leverages ResNet18-CBAM to extract features from fundus images and Optical Coherence Tomography(OCT)images,respectively,and conduct feature fusion to classify 11 common ophthalmic diseases.Moreover,Class Activation Mapping(CAM)is employed to visualize each mode to improve interpretability of M^(2)LC-Net.We conduct comprehensive experiments on realistic dataset collected from a Grade III Level A ophthalmology hospital in China,including 34,396 images of 11 disease labels.Experimental results demonstrate effectiveness of our proposed model M^(2)LC-Net.Compared with the stateof-the-art,various performance metrics have been improved significantly.Specifically,Cohen’s kappa coefficient κ has been improved by 3.21%,which is a remarkable improvement.

Exploration of Multi-modal Teaching Methods in Teaching of Agricultural Landscape Planning

Taking the teaching practice of agricultural landscape planning for example,this paper uses the multi-modal teaching idea for teaching design based on traditional lecture-style teaching,including multi-modal teaching materials,multi-modal teaching methods and multi-modal teaching evaluation. The results show that this method can effectively improve students' interest in learning,reinforce the theoretical basis of agricultural landscape planning theory,and improve agricultural landscape planning practical skills. It is the active exploration of multi-modal teaching model and useful complement to traditional classroom teaching.

Speed-up Multi-modal Near Duplicate Image Detection

Near-duplicate image detection is a necessary operation to refine image search results for efficient user exploration. The existences of large amounts of near duplicates require fast and accurate automatic near-duplicate detection methods. We have designed a coarse-to-fine near duplicate detection framework to speed-up the process and a multi-modal integra-tion scheme for accurate detection. The duplicate pairs are detected with both global feature (partition based color his-togram) and local feature (CPAM and SIFT Bag-of-Word model). The experiment results on large scale data set proved the effectiveness of the proposed design.

Modal identification of multi-degree-of-freedom structures based on intrinsic chirp component decomposition method

Modal parameter identification is a mature technology.However,there are some challenges in its practical applications such as the identification of vibration systems involving closely spaced modes and intensive noise contamination.This paper proposes a new time-frequency method based on intrinsic chirp component decomposition(ICCD)to address these issues.In this method,a redundant Fourier model is used to ameliorate border distortions and improve the accuracy of signal reconstruction.The effectiveness and accuracy of the proposed method are illustrated using three examples:a cantilever beam structure with intensive noise contamination or environmental interference,a four-degree-of-freedom structure with two closely spaced modes,and an impact test on a cantilever rectangular plate.By comparison with the identification method based on the empirical wavelet transform(EWT),it is shown that the presented method is effective,even in a high-noise environment,and the dynamic characteristics of closely spaced modes are accurately determined.

Research on Satellite Fault Diagnosis and Prediction Using Multi-modal Reasoning

Diagnosis and prediction of satellite fault are more difficult than that of other equipment due to the complex structure of satellites and the presence of multi excite sources of satellite faults. Generally, one kind of reasoning model can only diagnose and predict one kind of satellite faults. In this paper the author introduces an application of a new method using multi modal reasoning to diagnose and predict satellite faults. The method has been used in the development of knowledge based satellite fault diagnosis and recovery system (KSFDRS) successfully. It is shown that the method is effective.

Misuse of Teaching Gesture from the Perspective of Multi-modal Discourse Analysis and Its Solutions

Based on the teaching video of middle school English teachers, through observation and analysis, it puts forward the problem of less use, wrong use and abuse in the use of teachers' teaching gestures in middle school English teaching. And then it puts forward corresponding solutions from three aspects: concept, theory and practice. Hoping to provide further reference to the complementary role of teaching gesture and teaching discourse.

基于多模态融合的3DRes-ViT膝骨关节炎分类模型

针对膝骨关节炎的多分类准确率低,对膝骨关节图像特征提取不充分的问题,提出了基于多模态融合的3DRes-ViT网络模型。设计一种三维卷积神经网络(3D Convolutional Neural Networks,3D CNN),分别提取双回声稳态(DESS)和快速自旋回波(TSE)磁共振成像(Magnetic Resonance Imaging,MRI)序列的三维浅层特征。研究发现这两种信息具有互补性,将这些特征进行融合。通过高效通道注意力(Efficient Channel Attention,ECA)模块捕捉融合后特征通道间的依赖关系,并输入到Vision Transformer(ViT)编码器中,结合3DCNN和ViT的优势高效聚合两个模态的局部特征和全局特征。最后,ViT的输出再与二维卷积神经网络(2D CNN)提取的X光图像特征进行融合,以进一步提升分类性能。实验结果表明,本文方法在KOA四分类任务中表现优异,平均分类准确率达到了91.2%、平均精度为91.6%,F1分数为0.914,平均绝对误差降低至8.8%,显著提高了膝骨关节炎多分类的准确率。

基于视觉-语言预训练模型的零样本迁移学习方法综述

近年来随着人工智能(AI)技术在计算机视觉与自然语言处理等单模态领域表现出愈发优异的性能,多模态学习的重要性和必要性逐渐展现出来,其中基于视觉-语言预训练模型的零样本迁移(ZST)方法得到了国内外研究者的广泛关注。得益于预训练模型强大的泛化性能,使用视觉-语言预训练模型不仅能提高零样本识别任务的准确率,而且能够解决部分传统方法无法解决的零样本下游任务问题。对基于视觉-语言预训练模型的ZST方法进行概述,首先介绍了零样本学习(FSL)的传统方法,并对其主要形式加以总结;然后阐述了基于视觉-语言预训练模型的ZST和FSL的区别及其可以解决的新任务;其次介绍了基于视觉-语言预训练模型的ZST方法在样本识别、目标检测、语义分割、跨模态生成等下游任务中的应用情况;最后对现有的基于视觉-语言预训练模型的ZST方法存在的问题进行分析并对未来的研究方向进行展望。

基于NSGA-Ⅱ传感位置优化的曲面重构及误差补偿方法

通过优化光纤布拉格光栅形状传感技术中传感点位置和补偿重构结果来提高薄层合金板三维形状重构精度。通过ANSYS workbench建立合金板仿真模型,提取应变和位移模态振型,根据模态置信准则、转换矩阵稳定性和模态振型相似性分别设计了三个目标函数,采用快速和精英机制的多目标遗传算法优化传感器位置。将镍钛合金板弯曲成不同曲率半径的弧形,利用光纤布拉格光栅中心波长漂移量和线性插值算法计算得出不同形状下的结构应变,重构合金板形状,均方根误差和最大误差相较于单目标优化算法分别减小30%和15%。利用粒子群优化径向基函数神经网络算法拟合误差与位移的关系实现误差补偿,均方根误差和最大误差比无补偿时分别减小了90%和70%,最大相对百分比误差仅为5%,提高了三维形状重构算法精度。