多模态融合下宠-主情感交流产品交互设计研究

INTERACTION DESIGN FOR PETS’AND PET OWNERS’EMOTIONAL COMMUNICATION PRODUCTS UNDER MULTIMODAL FUSION

针对宠-主交流产品情绪识别方式缺失或单一,宠-主情绪状态难以及时掌握把控,进而造成宠-主情感交流障碍的问题,提出多模态计算框架下的宠-主情感交流产品双向交互设计原则,实现对宠-主情绪精准识别,交互反馈,以促进宠-主情感交流。在现有人类情绪识别的音频分析与计算机视觉跨模态融合基础上,针对猫咪意象需求的模糊性与主人情感需求的多样性,联结宠-主情感与其对应需求,提出多模态视域下宠物情绪识别新方法和宠-主情感交流产品双向交互设计模型。面向宠-主情感交流的多模态情感交互模型,通过情绪识别、需求确定、交互反馈,可以提升宠物产品智能化,准确把握宠-主情绪,促进宠-主沟通的良性循环。

To address the problem of missing or single way of emotion recognition in pet and pet owner communication products,which makes it difficult to grasp and control the emotional state of pet and pet owner in time and thus causes obstacles to the emotional communication of pet and pet owner,we propose a bidirectional interaction design method for pet and pet owner emotional communication products under a multimodal computing framework to achieve accurate recognition of pet’s and pet owner’s emotion and interactive feedback to facilitate the emotional communication.Based on the existing cross-modal fusion of audio analysis and computer vision for human emotion recognition,we propose a new method for pet emotion recognition and a two-way interaction design model for pet and pet owner emotion communication products in a multimodal perspective,in response to the ambiguity of cat imagery needs and the diversity of owner emotion needs,and linking pet and pet owner emotions with their corresponding needs.The multimodal emotional interaction model for pets’and pet owners’emotional communication can promote a virtuous cycle of pet owner communication through emotion recognition,demand determination,and interaction feedback.Therefore,building a multimodal interaction design model for pets’and pet owners’emotional communication products,can enhance the intelligence of pet products,accurately grasp pets’and pet owners’emotions,and eliminate communication barriers between them.