基于视-触跨模态感知的智能导盲系统

An intelligent blind guidance system based on visual-touch cross-modal perception

盲人活动援助是盲人日常生活的重要组成部分。这些技术大多用于帮助盲人导航和躲避障碍物,很少有研究将地面信息转换成一种给用户直观感受的触觉信息。为了满足上述需求,本文提出了一种可以提供触觉反馈的盲人辅助地面识别智能导盲杖系统。试图利用深度生成对抗训练的方法来产生振动触觉刺激,使用改进的DiscoGAN训练了我们的端到端生成网络。为了训练我们的网络,构建了视触跨模态数据集GroVib。通过上机实验和实物实验来评估方案的可行性,通过上机实验结果表明参与者通过触觉识别地面的准确率为84.7%,触觉的平均真实感受得分为71.3,在真实场景实验中,参与者只需平均3.35次尝试就可以根据触觉反馈来识别地面。

Blind mobility aid is a primary part of the daily life for blind people.Most of these technologies are used to help them navigate and avoid obstacles,and few researches have been done to convert ground information into tactile sensation that gives the user an intuitive feeling.To meet the above requirements,we proposed an intelligent guided rod system,which can provide tactile feedback to assist the blind to recognize ground information.We attempted to generate the vibrotactile stimuli leveraging the power of deep generative adversarial training.Specifically,we used an improved DiscoGAN training an end-to-end generated network.To train the network,we also built GroVib,a visual touch cross-modal dataset.We set up computer experiments and physical experiments to evaluate the feasibility of the scheme.The results of computer experiments showed that the accuracy rate of the participants in recognizing the ground by tactile sense was 84.7%,and the average real feeling score of tactile sense was 71.3.In real scene experiments,the participants needed only 3.25 times of attempts on average to recognize the ground based on tactile feedback.