基于宽度学习方法的多模态信息融合

Multi-modal information fusion based on broad learning method

多模态机器学习通过有效学习各个模态的丰富特征来解决不同模态数据的融合问题。考虑到模态间的差异性,基于宽度学习方法提出了一个能够学习和融合两种模态特征的框架,首先利用宽度学习方法分别提取不同模态的抽象特征,然后将高维特征表示在同一个特征空间进行相关性学习,并通过非线性融合得到最后的特征表达,输入分类器进行目标识别。相关实验建立在康奈尔大学抓取数据集和华盛顿大学RGB-D数据集上,实验结果验证了相比于传统的融合方法,所提出的方法具有更好的稳定性和快速性。

Multi-modal machine learning solves the fusion problem that arises in data with different modalites by effectively learning their rich characteristics.Considering the differences between various modalities,we propose a framework that can learn and fuse two kinds of modal characteristics based on the broad learning method.This method first extracts different abstract characteristics,then represents the high-dimension features in the same space to determine their correlation.We obtain a final representation of these characteristics by nonlinear fusion and inputs these characteristics into a classifier for target recognition.Relevant experiments are conducted on the Cornell Grasping Dataset and the Washington RGB-D Object Dataset,and our experimental results confirm that,compared with traditional fusion methods,the proposed algorithm has greater stability and rapidity.