Towards engineering a portable platform for laparoscopic pre-training in virtual reality with haptic feedback

Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.

Real Time and Efficient Haptic Feedback Framework for Deformable Objects

On account of the high computational speed required for force feedback and the detailed graphic required for deformation feedback, this paper presents a real time and efficient haptic feedback framework by subdividing the area of interest on a relatively coarse triangular mesh. When a contact is detected, a portion of the colliding surface is subdivided and a particle system produced, which is used for deformation based on the approximate velocity cone theory. Simultaneously, a spring\|damper model is used ...

Effect of haptic feedback on a virtual lab about friction

Background With the increase in recent years of the utilization of multimedia devices in education,new haptic devices for education have been gradually adopted and developed.As compared with visual and auditory channels,the development of applications with a haptic channel is still in the initial stages.For example,it is unclear how force feedback influences an instructional effect of an educational application and the subjective feeling of users.Methods In this study,we designed an educational application with a haptic device(Haply)to explore the effects of force feedback on self learning.Subjects in an experiment group used a designed application to study friction by themselves using force feedback,whereas subjects in a control group studied the same knowledge without force feedback.A post-test and questionnaire were designed to assess the learning outcomes.Results/Conclusions The experimental result indicates that force feedback is beneficial to an educational application,and using a haptic device can improve the effect of the application and motivate students.

基于主客观数据的汽车人机交互界面表面触觉反馈方法

针对汽车用户界面虚拟按钮缺乏触觉反馈或触觉反馈感受单一的问题,采集3种真实物理按钮(切换开关、自锁按钮、复位按钮)数据并提取按钮特征,基于手指按压力大小分段实现了不同的振动反馈。提出一种重现按钮顺应性的触觉渲染方法,根据用户主观感受,验证了其相比于顺应性错觉方法在顺应性、用户愉悦性等方面的提升。进行触觉反馈匹配实验,实验者根据触觉感受将3种虚拟按钮与物理按钮类型进行匹配,准确率高达94.33%,验证了提出的虚拟按钮振动触觉反馈对不同物理按钮独特感知特性恢复的有效性,用户可以在无视觉需求的情况下辨别操纵的虚拟按钮类型和状态。

沉浸式虚拟现实中加入触觉反馈对学习效果的影响——来自消防安全教育的证据

随着虚拟现实硬件设备的快速普及和使用成本的降低,沉浸式虚拟现实(IVR)在教学中的应用逐渐增多,但其研究大多关注视觉和听觉体验,对引入触觉反馈后在学习效果提升上的有效性仍缺乏证据。消防安全教育是IVR的典型应用场景,探究在IVR消防安全教育系统中加入触觉反馈(力反馈和热感)对学习者学业成绩、学习动机和认知负荷的影响,有助于为优化IVR教学应用提供依据。基于该情境的对照实验结果表明:IVR能够显著促进学习者对程序性知识、事实性知识和概念性知识的学习,并且加入触觉反馈后的知识增益更高,更加有利于程序性知识的习得;触觉反馈的加入显著降低了学习者的认知负荷,尤其是外在认知负荷,但对学习者学习动机的影响并不显著。研究结论为多媒体学习中触觉通道的整合提供了新的证据,拓展了原有多媒体学习的认知理论框架,有助于指导IVR教育资源的设计、开发及应用。

压觉-滑觉-粗糙度多模态指尖触觉刺激装置研究

为解决手康复训练中指尖触觉刺激装置刺激形式单一、体积与质量过大、刺激效果不佳的问题,提出了一种压觉-滑觉-粗糙度多模态指尖触觉刺激装置。首先,分析人体指尖曲面特征,设计多接触面的气动压觉反馈装置,包括中央、指尖、两侧共计4个气室;然后,设计电极阵列模块,采用电刺激实现滑觉、粗糙度的反馈,建立人体阻抗模型确定供电电压等电刺激参数,并通过仿真验证;最后,采用柔性材料分别打印制作气动压觉执行器和电极阵列,集成的触觉反馈装置质量为14.8 g,尺寸为40 mm长、30 mm宽。实验结果表明,气动压觉反馈装置的线性度较高、迟滞小,单腔力反馈最大为12 N;电刺激装置的粗糙度识别准确率达到100%,滑觉粗糙识别率超过80%。设计的装置可以同时提供给用户指尖精确的压觉、滑觉以及粗糙度的触觉反馈,并且质量轻、体积小、反馈力大,对多模态触觉反馈装置具有一定的参考价值,有应用于脑卒中患者康复训练的潜力。

柔性触觉反馈驱动器研究进展

触觉交互界面作为用户与系统之间的重要沟通窗口,不仅能够满足用户对简单化、自然化的人机界面的需求,也是实现直接、高效完成任务的关键因素。触觉反馈驱动器作为触觉交互过程中的核心部件,能够模拟或复现多种形式的触觉信息,提高用户在操作过程中的临场感与真实感。近年来,柔性触觉反馈驱动器凭借着其轻量、柔顺、适应性强等特点,在人机交互场景中展现了广阔的应用前景与价值。首先,根据柔性触觉反馈驱动器对皮肤刺激形式的不同分为电刺激、机械刺激、热刺激3类,并进一步重点阐述了各类驱动器的最新研究进展,主要包括了各类驱动器的驱动原理、结构设计、反馈形式等。进一步地,介绍了基于柔性触觉反馈驱动器所开发的触觉反馈设备在虚拟现实、遥操作、医疗等领域的应用。最后,对当前柔性触觉反馈驱动器所面临的问题与挑战进行了总结,并对未来的发展趋势进行了展望。

Virtual brain surgery simulation system based on haptic interaction

To improve the accuracy and interactivity of soft tissue delormatlon simulation, a new plate spring model based on physics is proposed. The model is parameterized and thus can be adapted to simulate different organs. Different soft tissues are modeled by changing the width, number of pieces, thickness, and length of a single plate spring. In this paper, the structural design, calcula- tion of soft tissue deformation and real-time feedback operations of our system are also introduced. To evaluate the feasibility of the system and validate the model, an experimental system of haptic in- teraction, in which users can use virtual hands to pull virtual brain tissues, is built using PHANTOM OMNI devices. Experimental results show that the proposed system is stable, accurate and promising for modeling instantaneous soft tissue deformation.

基于纹理图像和用户动作信息的纹理触觉再现方法研究

近年来,对虚拟表面纹理进行工具介导的振动触觉再现已经成为触觉领域的一个研究热点。针对已有的纹理触觉渲染方法还存在适用范围窄、泛化能力弱、交互真实感低等问题,以改进后的MelGAN为基本网络,构建了一种新的纹理触觉渲染模型。该模型以纹理图像和用户的实时动作信息为输入,在能够高保真地生成振动触觉信号的同时,对常见纹理图像拥有更好的泛化能力。此外,设计了一款具有动作信息实时采集和振动触觉表达功能的笔式装置。在采集了数据库之外的真实纹理表面的振动触觉信号后,比较了提出的模型与已有方法在信号生成方面的性能差异。结果表明,模型实现了最低的均方根误差(0.173),验证了其对未建模纹理进行触觉渲染的能力。最后,使用笔式装置进行了两项用户实验。平均6.01的主观相似度评分说明,即使是数据库之外的新纹理,模型也能为用户提供较高的纹理交互真实感。

主从遥操作机械手的阻尼触觉反馈

遥操作机械手因其独特的优势,已被广泛用于太空探索、深海探测、核设施维护和医疗手术。然而,随着任务复杂性和安全要求的提高,要求遥操作机械手具有更好的避障能力。针对此问题,提出一种新的基于可变阻尼的触觉反馈方法,为操作者提供阻尼力以感知远程环境中的障碍物,从而完成避障任务。在此基础上,提出避障阻尼模型,以解决传统的触觉方法由于考虑因素不全面而导致的不准确问题。该模型不仅考虑了传统的距离因素,还根据速度方向的新因素预测了遥操作机械手的碰撞情况,以在线调整阻尼值。实验结果表明:与没有触觉反馈的远程操作相比,该方法能够有效地提高远程操作的安全性和人类操作员的工作效率。

基于形状记忆合金的可穿戴触觉设备设计

可穿戴触觉设备设计的难点之一是装备体积、重量与触觉体验之间的平衡。形状记忆合金以其重量轻、体积小、柔软灵活及能提供多样化触觉反馈类型的特性,成为解决这一问题的潜在方案。本研究针对目前研究较少的“轻抚”类触觉反馈,结合视觉障碍人士的触觉导航需求和大众用户的情感调节需求,分别设计了基于形状记忆合金的可穿戴触觉导航设备和情绪调节手环原型。通过这两个原型示例,本文旨在揭示形状记忆合金在可穿戴触觉设备领域的应用前景,为未来人机交互技术的发展提供新的参考和思路。

Obstacle recognition using a haptic device

A haptic device is proposed which gives the user feedback information on their location and orientation of the obstacle through the mobile robot that detects the obstacle in an environment where the user cannot see.Mobile robot recognizes the exact position of the obstacle through configuring the nested ultrasonic sensor and giving feedback information to the haptic device.The haptic device consisting of five vibration motors can realize the haptic through the vibration of user's finger using the position information of the obstacle received feedback.In addition,it has high accuracy to recognize the surrounding environment and realizes the various situations with the fuzzy controller and the nested ultrasonic sensors.

The opinion and experience of surgeons with laparoscopic bowel grasper haptics

Background: In order to develop new and better laparoscopic bowel instruments, which reduces patient risks, the opinions and experience that surgeons have with current laparoscopic bowel grasper haptics is important. In this study we explored this by means of a questionnaire. Method: A total of 386 online- questionnaires, were sent to laparoscopic surgeons working in European hospitals. They were all members of the European Association of Endoscopic Surgery and perform laparoscopic obesities or bowel surgery. Surgeons where divided into different age and experience groups. Results: A total of 174 completely filled out forms were analyzed. In total, 16% of the surgeons cannot prevent damage when they pinch too hard, although they (10%) might have seen or felt it. Seven percent of the respondents were not able to see or feel tissue slippage. Whereas 31% can see or feel slippage they cannot do anything to prevent it. Overall, most of the respondents would appreciate technical changes in the laparoscopic bowel graspers to reduce tissue damage. Of all the respondents, 79% maintain that it is necessary to have a new laparoscopic grasper with augmented feedback. The majority of the respondents (77%) would like to have tactile feedback as an indication of the level of pinch force. There are not many differences in the opinions of surgeons at different skill levels. Conclusion: From the results of the questionnaire and the other comments made by respondents it is evident that research and developments in the field of new laparoscopic graspers should continue.

基于虚拟现实与触觉反馈技术的腰硬联合阻滞教学平台的研发应用

目的本研究旨在开发一个基于虚拟现实联合触觉反馈技术的腰硬联合阻滞教学平台,并对其进行教学评估。方法依据志愿者的腰部CT和MRI数据,采用MIMICS、3D MAX和UNITY 3D软件对人体模型进行三维重建,包括腰部脊柱、神经和肌肉组织以及手术环境,并对腰部各层次进行力反馈赋值。20名学员进行30次虚拟训练,由2名麻醉医师运用GRS、Checklist量表评估平台的易上手程度及学员的掌握水平,应用调查问卷获得学员的教学反馈。结果随着培训次数的增加,10次训练后,学员的操作时间稳定在(2.4±0.2)min。8次训练后,GRS评分趋于稳定,10次训练后Checklist评分也开始趋于稳定。调查问卷结果显示,学员认为该平台具有突破黄韧带的真实感、较强的可重复性和促进解剖识别的优点,80%的学员对平台感到满意。结论该教学平台可提高学员穿刺的流畅度和细节掌握能力,可作为椎管穿刺教学的一种有效方式。

Robot Position Control Using Force Information for Cooperative Work in Remote Robot Systems with Force Feedback

This paper proposes robot position control using force information for cooperative work between two remote robot systems with force feedback in each of which a user operates a remote robot by using a haptic interface device while observing work of the robot with a video camera. We also investigate the effect of the proposed control by experiment. As cooperative work, we deal with work in which two robots carry an object together. The robot position control using force information finely adjusts the position of the robot arm to reduce the force applied to the object. Thus, the purpose of the control is to avoid large force so that the object is not broken. In our experiment, we make a comparison among the following three cases in order to clarify how to carry out the control effectively. In the first case, the two robots are operated manually by a user with his/her both hands. In the second case, one robot is operated manually by a user, and the other robot is moved automatically under the proposed control. In the last case, the object is carried directly by a human instead of the robot which is operated by the user in the second case. As a result, experimental results demonstrate that the control can help each system operated manually by the user to carry the object smoothly.

触觉反馈技术在汽车信息系统中的应用

近年来汽车的智能网联化程度日益提高,现代车辆已经转变为拥有导航系统、娱乐系统、气候控制系统、车辆性能系统等信息系统的可移动计算机平台,驾驶员在车内可以访问和交互的信息量显著增加。然而在驾驶时操作功能繁多的汽车信息系统会分散司机大量的视觉和听觉注意力,造成分心驾驶,增加了发生交通事故的可能。驾驶时使用触觉模态可以与视听模态形成互补,减轻驾驶员视听负担,提高驾驶安全性、交互可靠性、使用舒适性。在总结触觉反馈技术在汽车中应用的发展历程的基础上,阐述了车内交互活动和触觉模态在车内可展现信息的具体种类;重点从车载触觉警告系统、车载触觉导航系统和车载触觉操作协助系统3个方向分析了触觉反馈技术在汽车信息系统中应用的研究进展,并分别总结了在车载警告系统、导航系统和操作协助系统中触觉激励的主要来源;分析总结相关文献,详述了汽车触觉信息系统对驾驶性能影响研究的实验环境、评估指标、评估方法和结果;基于现有技术和研究方法的不足,从触觉反馈激励信号的位置、模式和参数的选取、行为适应问题的研究、实验环境的改进、对影响触觉感知不同因素的考虑、多种汽车触觉信息系统的整合等方面,对未来汽车触觉信息系统的研究进行了讨论与展望。

基于触觉反馈的公交车驾驶员可穿戴产品设计

随着城镇化进程的加快,公共交通系统也在不断的完善,关注公共交通安全已成为一个社会热点。可穿戴产品的智能化日益显著,通过对触觉反馈的相关研究进行分析,旨在运用可穿戴产品帮助驾驶员降低认知负荷,运用可穿戴产品检测驾驶员的身体状况,向皮肤传递周围环境信号,实现更安全的驾驶过程。具体分析影响公交车驾驶安全的要素,总结得出公交车驾驶员的可穿戴产品设计的设计原则及应用。

手术机器人面临的一大挑战——力触觉反馈

机器人手术作为一种微创手术技术,由于其高度的精准性,已经得到了外科医生的广泛认可。但机器人手术最大的局限之一就是外科医生缺乏操作的"手感"(即力触觉反馈),增加了手术的不确定性和风险性,从而限制了手术机器人的进一步发展。文中从手术机器人力触觉反馈系统的组成、关键技术、以及当前的研究现状等几个方面,对该反馈系统进行了综述。力触觉反馈包括力反馈与触觉反馈。该系统的实现依赖力触觉的传感与再现,本文介绍了常用的力触觉传感器、再现设备及感觉模拟技术,并分析了各项技术的优缺点。对力触觉反馈系统在手术机器人的研究进展进行了概述。在汇总以前研究的基础上,对相关研究做了展望。