通过触觉编码和声音为视力障碍者提供远程虚拟陪伴

Remote Virtual Companion via Tactile Codes and Voices for The People With Visual Impairment

目的现有的人工视觉装置分为植入式装置和体外辅助装置两种,但它们都有一些不足之处。植入式装置需要手术植入、会造成不可恢复创伤;体外辅助装置指令相对简单、应用场景较为单一、过于依赖人工智能(AI)的判断不能提供足够的安全性。本文提出了一种将周边环境信息转化成头颈部触觉指令、并辅助以语音交互的系统,其有效性、安全性、信息量等均优于现有体外辅助技术,同时也具有低成本、低风险、适合多种生活和工作场景等优势。方法该系统借助最新的远程无线网络通讯技术、芯片技术,利用前方人员随身佩戴的微小型电子设备、摄像头和感应器,以及云端庞大的数据库和计算能力,后台人员可以实时、充分地远程(比如跨越城市)了解前方的现场景象、环境参数和人员状态等信息,通过对比云端数据库和内存数据库、AI辅助识别和人工综合分析,快速获得最合理的行动方案,并将行动指令及时传给前方人员,实现盲人导航功能。同时,也用语音互动对话提供人文关怀、情感寄托。结果本文首次提出了“远程虚拟陪伴概念”,并演示了相应的硬件和软件以及多种生活场景下的测试效果。除了可以实现基础的导航功能,比如帮助视觉障碍人群完成在超市购物、咖啡厅寻座、街道行走,以及完成更加复杂的拼图、日常娱乐打牌功能,还可以满足骑行等速度相对快的移动指令要求。结论实验结果表明,这种“远程虚拟陪伴”装置适用大量场景和需求,不仅可以用于视觉障碍人群出行、购物、娱乐,也可用于陪伴老人出行、辅助野外探险或旅行等,具有广泛的发展和应用前景。

Objective Existing artificial vision devices can be divided into two types:implanted devices and extracorporeal devices,both of which have some disadvantages.The former requires surgical implantation,which may lead to irreversible trauma,while the latter has some defects such as relatively simple instructions,limited application scenarios and relying too much on the judgment of artificial intelligence(AI)to provide enough security.Here we propose a system that has voice interaction and can convert surrounding environment information into tactile commands on head and neck.Compared with existing extracorporeal devices,our device can provide a larger capacity of information and has advantages such as lower cost,lower risk,suitable for a variety of life and work scenarios.Methods With the latest remote wireless communication and chip technologies,microelectronic devices,cameras and sensors worn by the user,as well as the huge database and computing power in the cloud,the backend staff can get a full insight into the scenario,environmental parameters and status of the user remotely(for example,across the city)in real time.In the meanwhile,by comparing the cloud database and in-memory database and with the help of AI-assisted recognition and manual analysis,they can quickly develop the most reasonable action plan and send instructions to the user.In addition,the backend staff can provide humanistic care and emotional sustenance through voice dialogs.Results This study originally proposes the concept of“remote virtual companion”and demonstrates the related hardware and software as well as test results.The system can not only achieve basic guide functions,for example,helping a person with visual impairment to shop in supermarkets,find seats at cafes,walk on the streets,construct complex puzzles,and play cards,but also can meet the demand for fast-paced daily tasks such as cycling.Conclusion Experimental results show that this“remote virtual companion”is applicable for various scenarios and demands.It can help blind people with their travels,shopping and entertainment,or accompany the elderlies with their trips,wilderness explorations,and travels.