Towards engineering a portable platform for laparoscopic pre-training in virtual reality with haptic feedback

Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.

Effect of haptic feedback on a virtual lab about friction

Background With the increase in recent years of the utilization of multimedia devices in education,new haptic devices for education have been gradually adopted and developed.As compared with visual and auditory channels,the development of applications with a haptic channel is still in the initial stages.For example,it is unclear how force feedback influences an instructional effect of an educational application and the subjective feeling of users.Methods In this study,we designed an educational application with a haptic device(Haply)to explore the effects of force feedback on self learning.Subjects in an experiment group used a designed application to study friction by themselves using force feedback,whereas subjects in a control group studied the same knowledge without force feedback.A post-test and questionnaire were designed to assess the learning outcomes.Results/Conclusions The experimental result indicates that force feedback is beneficial to an educational application,and using a haptic device can improve the effect of the application and motivate students.

基于主客观数据的汽车人机交互界面表面触觉反馈方法

针对汽车用户界面虚拟按钮缺乏触觉反馈或触觉反馈感受单一的问题,采集3种真实物理按钮(切换开关、自锁按钮、复位按钮)数据并提取按钮特征,基于手指按压力大小分段实现了不同的振动反馈。提出一种重现按钮顺应性的触觉渲染方法,根据用户主观感受,验证了其相比于顺应性错觉方法在顺应性、用户愉悦性等方面的提升。进行触觉反馈匹配实验,实验者根据触觉感受将3种虚拟按钮与物理按钮类型进行匹配,准确率高达94.33%,验证了提出的虚拟按钮振动触觉反馈对不同物理按钮独特感知特性恢复的有效性,用户可以在无视觉需求的情况下辨别操纵的虚拟按钮类型和状态。

压觉-滑觉-粗糙度多模态指尖触觉刺激装置研究

为解决手康复训练中指尖触觉刺激装置刺激形式单一、体积与质量过大、刺激效果不佳的问题,提出了一种压觉-滑觉-粗糙度多模态指尖触觉刺激装置。首先,分析人体指尖曲面特征,设计多接触面的气动压觉反馈装置,包括中央、指尖、两侧共计4个气室;然后,设计电极阵列模块,采用电刺激实现滑觉、粗糙度的反馈,建立人体阻抗模型确定供电电压等电刺激参数,并通过仿真验证;最后,采用柔性材料分别打印制作气动压觉执行器和电极阵列,集成的触觉反馈装置质量为14.8 g,尺寸为40 mm长、30 mm宽。实验结果表明,气动压觉反馈装置的线性度较高、迟滞小,单腔力反馈最大为12 N;电刺激装置的粗糙度识别准确率达到100%,滑觉粗糙识别率超过80%。设计的装置可以同时提供给用户指尖精确的压觉、滑觉以及粗糙度的触觉反馈,并且质量轻、体积小、反馈力大,对多模态触觉反馈装置具有一定的参考价值,有应用于脑卒中患者康复训练的潜力。

柔性触觉反馈驱动器研究进展

触觉交互界面作为用户与系统之间的重要沟通窗口,不仅能够满足用户对简单化、自然化的人机界面的需求,也是实现直接、高效完成任务的关键因素。触觉反馈驱动器作为触觉交互过程中的核心部件,能够模拟或复现多种形式的触觉信息,提高用户在操作过程中的临场感与真实感。近年来,柔性触觉反馈驱动器凭借着其轻量、柔顺、适应性强等特点,在人机交互场景中展现了广阔的应用前景与价值。首先,根据柔性触觉反馈驱动器对皮肤刺激形式的不同分为电刺激、机械刺激、热刺激3类,并进一步重点阐述了各类驱动器的最新研究进展,主要包括了各类驱动器的驱动原理、结构设计、反馈形式等。进一步地,介绍了基于柔性触觉反馈驱动器所开发的触觉反馈设备在虚拟现实、遥操作、医疗等领域的应用。最后,对当前柔性触觉反馈驱动器所面临的问题与挑战进行了总结,并对未来的发展趋势进行了展望。

主从遥操作机械手的阻尼触觉反馈

遥操作机械手因其独特的优势,已被广泛用于太空探索、深海探测、核设施维护和医疗手术。然而,随着任务复杂性和安全要求的提高,要求遥操作机械手具有更好的避障能力。针对此问题,提出一种新的基于可变阻尼的触觉反馈方法,为操作者提供阻尼力以感知远程环境中的障碍物,从而完成避障任务。在此基础上,提出避障阻尼模型,以解决传统的触觉方法由于考虑因素不全面而导致的不准确问题。该模型不仅考虑了传统的距离因素,还根据速度方向的新因素预测了遥操作机械手的碰撞情况,以在线调整阻尼值。实验结果表明:与没有触觉反馈的远程操作相比,该方法能够有效地提高远程操作的安全性和人类操作员的工作效率。

基于形状记忆合金的可穿戴触觉设备设计

可穿戴触觉设备设计的难点之一是装备体积、重量与触觉体验之间的平衡。形状记忆合金以其重量轻、体积小、柔软灵活及能提供多样化触觉反馈类型的特性,成为解决这一问题的潜在方案。本研究针对目前研究较少的“轻抚”类触觉反馈,结合视觉障碍人士的触觉导航需求和大众用户的情感调节需求,分别设计了基于形状记忆合金的可穿戴触觉导航设备和情绪调节手环原型。通过这两个原型示例,本文旨在揭示形状记忆合金在可穿戴触觉设备领域的应用前景,为未来人机交互技术的发展提供新的参考和思路。

Obstacle recognition using a haptic device

A haptic device is proposed which gives the user feedback information on their location and orientation of the obstacle through the mobile robot that detects the obstacle in an environment where the user cannot see.Mobile robot recognizes the exact position of the obstacle through configuring the nested ultrasonic sensor and giving feedback information to the haptic device.The haptic device consisting of five vibration motors can realize the haptic through the vibration of user's finger using the position information of the obstacle received feedback.In addition,it has high accuracy to recognize the surrounding environment and realizes the various situations with the fuzzy controller and the nested ultrasonic sensors.

基于纹理图像和用户动作信息的纹理触觉再现方法研究

近年来,对虚拟表面纹理进行工具介导的振动触觉再现已经成为触觉领域的一个研究热点。针对已有的纹理触觉渲染方法还存在适用范围窄、泛化能力弱、交互真实感低等问题,以改进后的MelGAN为基本网络,构建了一种新的纹理触觉渲染模型。该模型以纹理图像和用户的实时动作信息为输入,在能够高保真地生成振动触觉信号的同时,对常见纹理图像拥有更好的泛化能力。此外,设计了一款具有动作信息实时采集和振动触觉表达功能的笔式装置。在采集了数据库之外的真实纹理表面的振动触觉信号后,比较了提出的模型与已有方法在信号生成方面的性能差异。结果表明,模型实现了最低的均方根误差(0.173),验证了其对未建模纹理进行触觉渲染的能力。最后,使用笔式装置进行了两项用户实验。平均6.01的主观相似度评分说明,即使是数据库之外的新纹理,模型也能为用户提供较高的纹理交互真实感。

Virtual brain surgery simulation system based on haptic interaction

To improve the accuracy and interactivity of soft tissue deformation simulation,a new plate spring model based on physics is proposed.The model is parameterized and thus can be adapted to simulate different organs.Different soft tissues are modeled by changing the width,number of pieces,thickness,and length of a single plate spring.In this paper,the structural design,calculation of soft tissue deformation and real-time feedback operations of our system are also introduced.To evaluate the feasibility of the system and validate the model,an experimental system of haptic interaction,in which users can use virtual hands to pull virtual brain tissues,is built using PHANTOM OMNI devices.Experimental results show that the proposed system is stable,accurate and promising for modeling instantaneous soft tissue deformation.

The opinion and experience of surgeons with laparoscopic bowel grasper haptics

Background: In order to develop new and better laparoscopic bowel instruments, which reduces patient risks, the opinions and experience that surgeons have with current laparoscopic bowel grasper haptics is important. In this study we explored this by means of a questionnaire. Method: A total of 386 online- questionnaires, were sent to laparoscopic surgeons working in European hospitals. They were all members of the European Association of Endoscopic Surgery and perform laparoscopic obesities or bowel surgery. Surgeons where divided into different age and experience groups. Results: A total of 174 completely filled out forms were analyzed. In total, 16% of the surgeons cannot prevent damage when they pinch too hard, although they (10%) might have seen or felt it. Seven percent of the respondents were not able to see or feel tissue slippage. Whereas 31% can see or feel slippage they cannot do anything to prevent it. Overall, most of the respondents would appreciate technical changes in the laparoscopic bowel graspers to reduce tissue damage. Of all the respondents, 79% maintain that it is necessary to have a new laparoscopic grasper with augmented feedback. The majority of the respondents (77%) would like to have tactile feedback as an indication of the level of pinch force. There are not many differences in the opinions of surgeons at different skill levels. Conclusion: From the results of the questionnaire and the other comments made by respondents it is evident that research and developments in the field of new laparoscopic graspers should continue.

基于虚拟现实与触觉反馈技术的腰硬联合阻滞教学平台的研发应用

目的本研究旨在开发一个基于虚拟现实联合触觉反馈技术的腰硬联合阻滞教学平台,并对其进行教学评估。方法依据志愿者的腰部CT和MRI数据,采用MIMICS、3D MAX和UNITY 3D软件对人体模型进行三维重建,包括腰部脊柱、神经和肌肉组织以及手术环境,并对腰部各层次进行力反馈赋值。20名学员进行30次虚拟训练,由2名麻醉医师运用GRS、Checklist量表评估平台的易上手程度及学员的掌握水平,应用调查问卷获得学员的教学反馈。结果随着培训次数的增加,10次训练后,学员的操作时间稳定在(2.4±0.2)min。8次训练后,GRS评分趋于稳定,10次训练后Checklist评分也开始趋于稳定。调查问卷结果显示,学员认为该平台具有突破黄韧带的真实感、较强的可重复性和促进解剖识别的优点,80%的学员对平台感到满意。结论该教学平台可提高学员穿刺的流畅度和细节掌握能力,可作为椎管穿刺教学的一种有效方式。

Robot Position Control Using Force Information for Cooperative Work in Remote Robot Systems with Force Feedback

This paper proposes robot position control using force information for cooperative work between two remote robot systems with force feedback in each of which a user operates a remote robot by using a haptic interface device while observing work of the robot with a video camera. We also investigate the effect of the proposed control by experiment. As cooperative work, we deal with work in which two robots carry an object together. The robot position control using force information finely adjusts the position of the robot arm to reduce the force applied to the object. Thus, the purpose of the control is to avoid large force so that the object is not broken. In our experiment, we make a comparison among the following three cases in order to clarify how to carry out the control effectively. In the first case, the two robots are operated manually by a user with his/her both hands. In the second case, one robot is operated manually by a user, and the other robot is moved automatically under the proposed control. In the last case, the object is carried directly by a human instead of the robot which is operated by the user in the second case. As a result, experimental results demonstrate that the control can help each system operated manually by the user to carry the object smoothly.

触觉反馈技术在汽车信息系统中的应用

近年来汽车的智能网联化程度日益提高,现代车辆已经转变为拥有导航系统、娱乐系统、气候控制系统、车辆性能系统等信息系统的可移动计算机平台,驾驶员在车内可以访问和交互的信息量显著增加。然而在驾驶时操作功能繁多的汽车信息系统会分散司机大量的视觉和听觉注意力,造成分心驾驶,增加了发生交通事故的可能。驾驶时使用触觉模态可以与视听模态形成互补,减轻驾驶员视听负担,提高驾驶安全性、交互可靠性、使用舒适性。在总结触觉反馈技术在汽车中应用的发展历程的基础上,阐述了车内交互活动和触觉模态在车内可展现信息的具体种类;重点从车载触觉警告系统、车载触觉导航系统和车载触觉操作协助系统3个方向分析了触觉反馈技术在汽车信息系统中应用的研究进展,并分别总结了在车载警告系统、导航系统和操作协助系统中触觉激励的主要来源;分析总结相关文献,详述了汽车触觉信息系统对驾驶性能影响研究的实验环境、评估指标、评估方法和结果;基于现有技术和研究方法的不足,从触觉反馈激励信号的位置、模式和参数的选取、行为适应问题的研究、实验环境的改进、对影响触觉感知不同因素的考虑、多种汽车触觉信息系统的整合等方面,对未来汽车触觉信息系统的研究进行了讨论与展望。

手术机器人面临的一大挑战——力触觉反馈

机器人手术作为一种微创手术技术,由于其高度的精准性,已经得到了外科医生的广泛认可。但机器人手术最大的局限之一就是外科医生缺乏操作的"手感"(即力触觉反馈),增加了手术的不确定性和风险性,从而限制了手术机器人的进一步发展。文中从手术机器人力触觉反馈系统的组成、关键技术、以及当前的研究现状等几个方面,对该反馈系统进行了综述。力触觉反馈包括力反馈与触觉反馈。该系统的实现依赖力触觉的传感与再现,本文介绍了常用的力触觉传感器、再现设备及感觉模拟技术,并分析了各项技术的优缺点。对力触觉反馈系统在手术机器人的研究进展进行了概述。在汇总以前研究的基础上,对相关研究做了展望。

力反馈柔性虚拟夹具辅助遥操作

针对未知环境遥操作的操作安全性与操作性能之间相互矛盾的问题,提出并实现一种力反馈柔性虚拟夹具方法.该方法通过引入柔性系数调节机器人倾向于期望运动方向的程度,将离线任务规划与操作者手动控制的优势相结合,在获得一定的操作性能的同时,具有避开离线任务规划路径中障碍的能力,保证了遥操作的安全性.在此基础上,将虚拟力反馈方法融合到柔性虚拟夹具中,进一步提高遥操作性能.正弦曲线跟踪实验结果表明,所提方法减小了平均跟踪误差和任务完成时间,保证了未知环境遥操作的安全性和操作性能.

腹腔镜手术中软组织按压仿真

精确可靠的软组织变形模型对虚拟手术至关重要,针对现有模型的不足,提出了一种等节距圆锥形螺旋弹簧模型,模拟对腹腔镜手术中软组织的按压。该模型吸收了弹簧-质点模型计算快速的优点,通过变形截止层控制变形范围,提升模型的稳定性和精确度。为验证模型各方面性能,首先分别与有限元模型和真实软组织对比变形效果,然后记录力反馈输出与变形响应时间并与弹簧-质点模型比较,最后邀请受试者评分。实验结果表明,模型变形精度高,变形与力反馈输出响应时间均在20ms以下,且普遍低于弹簧-质点模型响应时间。在评分实验中本模型所模拟的3种软组织各指标均分分别为9.2、9.0和9.0,普遍高于其他模型均分。

支持触觉力反馈的虚拟油泥造型系统

使用六自由度三维鼠标Spaceball 5000和触觉力反馈设备PHANTOM Desktop,设计了造型操作中的双手交互子任务分配模式,完成了触觉反馈力的计算,并基于Open Inventor和Qt应用程序框架,开发了设备的用户接口,实现了带触觉力反馈功能的虚拟油泥造型系统,再现了传统油泥建模过程中通过力触觉控制造型过程的方法。最后,在实际产品开发过程中进行了应用验证,效果良好。