Exploring the effect of fingertip aero-haptic feedforward cues in directing eyes-free target acquisition in VR

Background The sense of touch plays a crucial role in interactive behavior within virtual spaces,particularly when visual attention is absent.Although haptic feedback has been widely used to compensate for the lack of visual cues,the use of tactile information as a predictive feedforward cue to guide hand movements remains unexplored and lacks theoretical understanding.Methods This study introduces a fingertip aero-haptic rendering method to investigate its effectiveness in directing hand movements during eyes-free spatial interactions.The wearable device incorporates a multichannel micro-airflow chamber to deliver adjustable tactile effects on the fingertips.Results The first study verified that tactile directional feedforward cues significantly improve user capabilities in eyes-free target acquisition and that users rely heavily on haptic indications rather than spatial memory to control their hands.A subsequent study examined the impact of enriched tactile feedforward cues on assisting users in determining precise target positions during eyes-free interactions,and assessed the required learning efforts.Conclusions The haptic feedforward effect holds great practical promise in eyeless design for virtual reality.We aim to integrate cognitive models and tactile feedforward cues in the future,and apply richer tactile feedforward information to alleviate users'perceptual deficiencies.

Resource allocation for D2D-assisted haptic communications

Haptic communications is recognized as a promising enabler of extensive services by enabling real-time haptic control and feedback in remote environments,e.g.,teleoperation and autonomous driving.Considering the strict transmission requirements on reliability and latency,Device-to-Device(D2D)communications is introduced to assist haptic communications.In particular,the teleoperators with poor channel quality are assisted by auxiliaries,and each auxiliary and its corresponding teleoperator constitute a D2D pair.However,the haptic interaction and the scarcity of radio resources pose severe challenges to the resource allocation,especially facing the sporadic packet arrivals.First,the contentionbased access scheme is applied to achieve low-latency transmission,where the resource scheduling latency is omitted and users can directly access available resources.In this context,we derive the reliability index of D2D pairs under the contention-based access scheme,i.e.,closed-loop packet error probability.Then,the reliability performance is guaranteed by bidirectional power control,which aims to minimize the sum packet error probability of all D2D pairs.Potential game theory is introduced to solve the problem with low complexity.Accordingly,a distributed power control algorithm based on synchronous log-linear learning is proposed to converge to the optimal Nash Equilibrium.Experimental results demonstrate the superiority of the proposed learning algorithm.

Towards engineering a portable platform for laparoscopic pre-training in virtual reality with haptic feedback

Background Laparoscopic surgery is a surgical technique in which special instruments are inserted through small incision holes inside the body.For some time,efforts have been made to improve surgical pre training through practical exercises on abstracted and reduced models.Methods The authors strive for a portable,easy to use and cost-effective Virtual Reality-based(VR)laparoscopic pre-training platform and therefore address the question of how such a system has to be designed to achieve the quality of today's gold standard using real tissue specimens.Current VR controllers are limited regarding haptic feedback.Since haptic feedback is necessary or at least beneficial for laparoscopic surgery training,the platform to be developed consists of a newly designed prototype laparoscopic VR controller with haptic feedback,a commercially available head-mounted display,a VR environment for simulating a laparoscopic surgery,and a training concept.Results To take full advantage of benefits such as repeatability and cost-effectiveness of VR-based training,the system shall not require a tissue sample for haptic feedback.It is currently calculated and visually displayed to the user in the VR environment.On the prototype controller,a first axis was provided with perceptible feedback for test purposes.Two of the prototype VR controllers can be combined to simulate a typical both-handed use case,e.g.,laparoscopic suturing.A Unity based VR prototype allows the execution of simple standard pre-trainings.Conclusions The first prototype enables full operation of a virtual laparoscopic instrument in VR.In addition,the simulation can compute simple interaction forces.Major challenges lie in a realistic real-time tissue simulation and calculation of forces for the haptic feedback.Mechanical weaknesses were identified in the first hardware prototype,which will be improved in subsequent versions.All degrees of freedom of the controller are to be provided with haptic feedback.To make forces tangible in the simulation,characteristic values need to be determined using real tissue samples.The system has yet to be validated by cross-comparing real and VR haptics with surgeons.

Effectiveness of anti-psychiatric treatment on visual and haptic perceptual disorder for a patient with Alzheimer’s disease: A case report

BACKGROUND Perception is frequently impaired in patients with Alzheimer’s disease(AD).Several patients exhibit visual or haptic hallucinations.CASE SUMMARY A 71-year-old Chinese man presented with visual and haptic hallucinations he had been experiencing for 2 weeks.The clinical manifestations were the feeling of insects crawling and biting the limbs and geison.He looked for the insects while itching and scratching,which led to skin breakage on the limbs.He was treated with topical and anti-allergic drugs in several dermatology departments without any significant improvement.After admission,the patient was administered risperidone(0.5 mg)and duloxetine(2 mg/day).One week later,the dose of risperidone was increased to 2 mg/day,and that of duloxetine was increased to 60 mg/day.After 2 weeks of treatment,the patient’s sensation of insects crawling and biting disappeared,and his mood stabilized.CONCLUSION This patient manifested psychiatric behavioral symptoms caused by AD brain atrophy.It was important to re-evaluate the patient’s cognitive-psychological status when the patient repeatedly went to the hospital for treatment.Follow-up attention to cognitive function and the consideration of perceptual deficits as early manifestations of AD should be considered.

Haptic display for virtual reality: progress and challenges

Immersion, interaction, and imagination are three features of virtual reality (VR). Existing VR systems possess fairly realistic visual and auditory feedbacks, and however, are poor with haptic feedback, by means of which human can perceive the physical world via abundant haptic properties. Haptic display is an interface aiming to enable bilateral signal communications between human and computer, and thus to greatly enhance the immersion and interaction of VR systems. This paper surveys the paradigm shift of haptic display occurred in the past 30 years, which is classified into three stages, including desktop haptics, surface haptics, and wearable haptics. The driving forces, key technologies and typical applications in each stage are critically reviewed. Toward the future high-fidelity VR interaction, research challenges are highlighted concerning handheld haptic device, multimodal haptic device, and high fidelity haptic rendering. In the end, the importance of understanding human haptic perception for designing effective haptic devices is addressed.

Real Objects Understanding Using 3D Haptic Virtual Reality for E-Learning Education

In the past two decades,there has been a lot of work on computer vision technology that incorporates many tasks which implement basic filtering to image classification.Themajor research areas of this field include object detection and object recognition.Moreover,wireless communication technologies are presently adopted and they have impacted the way of education that has been changed.There are different phases of changes in the traditional system.Perception of three-dimensional(3D)from two-dimensional(2D)image is one of the demanding tasks.Because human can easily perceive but making 3D using software will take time manually.Firstly,the blackboard has been replaced by projectors and other digital screens so such that people can understand the concept better through visualization.Secondly,the computer labs in schools are now more common than ever.Thirdly,online classes have become a reality.However,transferring to online education or e-learning is not without challenges.Therefore,we propose a method for improving the efficiency of e-learning.Our proposed system consists of twoand-a-half dimensional(2.5D)features extraction using machine learning and image processing.Then,these features are utilized to generate 3D mesh using ellipsoidal deformation method.After that,3D bounding box estimation is applied.Our results show that there is a need to move to 3D virtual reality(VR)with haptic sensors in the field of e-learning for a better understanding of real-world objects.Thus,people will have more information as compared to the traditional or simple online education tools.We compare our result with the ShapeNet dataset to check the accuracy of our proposed method.Our proposed system achieved an accuracy of 90.77%on plane class,85.72%on chair class,and car class have 72.14%.Mean accuracy of our method is 70.89%.

A Review of Haptic Technology Applications in Healthcare

Regaining lost abilities via the formation of new neural pathways as a result of extensive practice and retraining is a key feature of the rehabilitation process. However, patient boredom is a big issue in rehabilitation owing to the extensive practice that is necessary. Awareness of the value of haptic (touch-enabled) interfaces in computer applications has grown because to the emergence of games with buzzing joysticks and consumer electronics with touchpads. Some forms of haptics, however, are paving the way for cutting-edge computer-based rehabilitation solutions, such as “serious games,” which are only one use of haptics’ potential to enhance clinical treatment. In this paper, we aim to review the methods and technologies of haptic feedback that are used for medical treatment and rehabilitation. Developing and innovating better haptic feedback technology has the potential to expand and improve access to virtual healthcare such as the doctor being able to “feel” a patient’s tumor from miles away by being paired with robotic devices.

Obstacle recognition using a haptic device

A haptic device is proposed which gives the user feedback information on their location and orientation of the obstacle through the mobile robot that detects the obstacle in an environment where the user cannot see.Mobile robot recognizes the exact position of the obstacle through configuring the nested ultrasonic sensor and giving feedback information to the haptic device.The haptic device consisting of five vibration motors can realize the haptic through the vibration of user's finger using the position information of the obstacle received feedback.In addition,it has high accuracy to recognize the surrounding environment and realizes the various situations with the fuzzy controller and the nested ultrasonic sensors.

QoE Assessment in Haptic Teleoperation Systems： Position-Position versus Position-Force

In this paper, we make a comparison of haptic control schemes （position-position and position-force control schemes） for two haptic teleoperation systems by QoE （quality of experience） assessment. One is a remote control system with haptic media and video. In the system, we can write characters by controlling a haptic interface device at a remote place with another haptic interface device while watching video. The other is a remote instruction system with haptic media, video and voice. By using the system, a teacher can actively write characters and teach a student how to write the characters. The student can also write characters actively, and the teacher can confirm how the student writes the characters. We investigate the influences of network delay on QoE for the two systems with the two haptic control schemes.

基于Haptic技术的盲人路径诱导辅具应用研究现状

触觉(Haptic)技术信息表达是一种有效的人机交互模式和信息传递方式,并作为信息传递的重要通道用于盲人导航/路径诱导辅具领域,弥补了在特定条件下声音的缺点与不足。对盲人导航/路径诱导辅具总体情况进行了概述;结合空间认知介绍了基于Haptic技术的盲人导航/路径诱导辅具研究现状与应用;分析了基于Haptic技术的盲人导航/路径诱导辅具在今后的发展趋势和存在的问题,并讨论了应用前景。

Development and Evaluation of a 7-DOF Haptic Interface

With the development of human robot interaction technologies, haptic interfaces are widely used for 3 D applications to provide the sense of touch. These interfaces have been utilized in medical simulation, virtual assembly and remote manipulation tasks. However, haptic interface design and control are still critical problems to reproduce the highly sensitive touch sense of humans. This paper presents the development and evaluation of a7-DOF(degree of freedom) haptic interface based on the modified delta mechanism. Firstly, both kinematics and dynamics of the modified mechanism are analyzed and presented. A novel gravity compensation algorithm based on the physical model is proposed and validated in simulation. A haptic controller is proposed based on the forward kinematics and the gravity compensation algorithm. To evaluate the control performance of the haptic interface, a prototype has been implemented. Three kinds of experiments: gravity compensation, static response and force tracking are performed respectively. The experimental results show that the mean error of the gravity compensation is less than 0.7 N and the maximum continuous force along the axis can be up to 6 N. This demonstrates the good performance of the proposed haptic interface.

Virtual brain surgery simulation system based on haptic interaction

To improve the accuracy and interactivity of soft tissue delormatlon simulation, a new plate spring model based on physics is proposed. The model is parameterized and thus can be adapted to simulate different organs. Different soft tissues are modeled by changing the width, number of pieces, thickness, and length of a single plate spring. In this paper, the structural design, calcula- tion of soft tissue deformation and real-time feedback operations of our system are also introduced. To evaluate the feasibility of the system and validate the model, an experimental system of haptic in- teraction, in which users can use virtual hands to pull virtual brain tissues, is built using PHANTOM OMNI devices. Experimental results show that the proposed system is stable, accurate and promising for modeling instantaneous soft tissue deformation.

Testing New Solutions for Eco-Driving: Haptic Gas Pedals in Electric Vehicles

Reducing energy consumption has become a matter of increasing concern for electric vehicle owners. EcoDriver is a project funded by the European Commission, searching for new eco-driving solutions for reducing energy consumption in private and public transport. EcoDriver’s main purpose is to teach efficient driving strategies and facilitate drivers’ decision-making processes through several feedback modalities, in order to help increase driving efficiency and therefore reduce energy consumption. In the present study, the Full ecoDriver System combined with a haptic feedback gas pedal was tested in real driving conditions to give answers to some questions about its effectiveness, efficiency, workload and acceptability in an electric vehicle. The sample profile was composed by thirty young but experienced drivers. They had to drive around an open road track which allowed several possible scenarios such as curves, intersection or roundabout, speed limit changes and preceding vehicles. Average speed was registered on each lap, likewise other subjective measurements. The main results suggest that the efficiency benefits achieved while driving depend on the event type and the feedback modality provided. For instance haptic feedback seems to be especially indicated for roundabouts. In addition, the visual feedback provided by the FeDS nomadic device helps to save energy and learn eco-driving strategies. These outcomes indicate how several feedback modalities could facilitate the decision making process, changing driving behaviour, reducing energy consumption and increasing safety. These questions would help advance further research on eco-driving Intelligent Transport Systems and driving behaviour issues.

QoE Assessment of Group Synchronization Control Scheme with Prediction in Work Using Haptic Media

This paper proposes a group synchronization control scheme with prediction in work using haptic media. The scheme adjusts the output timing among multiple terminals and keeps the interactivity high. It outputs position information by predicting the future position later than the position included in the last-received information by a fixed amount of time. It also advances the output time of position information at each local terminal by the same amount of time. We deal with two different types of work using haptic media so as to demonstrate the effectiveness of the scheme. We assess the output quality of haptic media for the two types of work subjectively and objectively by Quality of Experience (QoE) assessment. We further clarify the relationship between subjective and objective assessment results.

Efficacy of scleral-fixated 3-looped haptics intraocular lens implantation for surgical management of microspherophakia

AIM:To evaluate the safety and efficacy of scleral-fixated 3-looped haptics intraocular lens(IOL)implantation for surgical management of microspherophakia.METHODS:A retrospective case series include 10 microspherophakic patients(15 eyes)who underwent lens removal plus a modified surgical treatment of scleral-fixated 3-looped haptics IOL implantation.The primary outcomes involved visual acuity,intraocular pressure(IOP).Secondary outcomes were spherical equivalent(SE),anterior chamber depth(ACD),corneal endothelial cell density and postoperative complications.RESULTS:After a postoperative follow-up of 17.60±15.44mo,improved visual outcomes can be observed.The uncorrected distance visual acuity(UCVA)log MAR improved from 1.54±0.59 preoperatively to 0.51±0.35 postoperatively(P=0.001),and best corrected visual acuity(BCVA)log MAR improved from 0.97±0.91 preoperatively to 0.24±0.23 postoperatively(P=0.003).Moreover,the SE decreased from-9.58±7.47D preoperatively to-0.65±2.21 D postoperatively(P<0.001).In terms of safety profile,the average IOP decreased from 21.10±12.94 mm Hg preoperatively to 14.03±3.57 mm Hg postoperatively(P=0.044),and the previously elevated IOP of three eyes decreased to the normal range.The ACD increased from 2.25±1.45 mm preoperatively to 3.35±0.39 mm postoperatively(P=0.017).The density of corneal endothelial cells did not change significantly after surgery(P=0.140).The posterior chamber IOLs were well centered and no severe complications were found.CONCLUSION:Lens removal plus the modified surgical treatment of scleral-fixated 3-looped haptics IOL implantation can help in improvement of visual acuity,which can be regarded as a relative safe method for the surgical management of microspherophakia.

Real-time haptic model for soft tissue deformation in surgery simulation

The modelling and simulation of deformable objects is a challenging topic in the field of haptic rendering between human and virtual environment.In this paper,a novel and efficient layered rhombus-chain-connected haptic deformation model based on physics is proposed for an excellent haptic rendering.During the modelling,the accumulation of relative displacements in every chain structure unit in each layer is equal to the deformation on the virtual object surface,and the resultant force of corresponding springs is equivalent to the external force.The layered rhombus-chain-connected model is convenient and fast to calculate,and can satisfy real-time requirement due to its simple nature.Simulation experiments in virtual human liver based on the proposed model are conducted,and the results demonstrate that our model provides stable and realistic haptic feeling in real time.Meanwhile,the display result is vivid.

Haptic Interface with Magnetic Field Sensitive Elastomer

Sense of touch is one of the important information from environment for human to live in daily life. Haptic interface is a hot topic in virtual reality but almost all of the devices focus on fingers and hands as targets. In this paper, we focus on the foot haptic device with magnetic flied sensitive elastomer （MSE）. We developed a haptic unit used as a magnetic field generator for MSE and contact point of foot haptic device. MSE samples mixed with 80 wt% carbonyl iron particles were prepared and evaluated with the developed magnet. Experimental results show that the mechanical property of the haptic unit can be modeled with the adjustable friction element. This property has a good advantage for the haptic unit.

Multi-dimensional force sensor for haptic interaction:a review

Haptic interaction plays an important role in the virtual reality technology,which let a person not only view the 3D virtual environment but also realistically touch the virtual environment.As a key part of haptic interaction,force feedback has become an essential function for the haptic interaction.Therefore,multi-dimensional force sensors are widely used in the fields of virtual reality and augmented reality.In this paper,some conventional multi-dimensional force sensors based on different measurement principles,such as resistive,capacitive,piezoelectric,are briefly introduced.Then the mechanical structures of the elastic body of multi-dimensional force sensors are reviewed.It is obvious that the performance of the multi-dimensional force sensor is mainly dependent upon the mechanical structure of elastic body.Furthermore,the calibration process of the force sensor is analyzed,and problems in calibration are discussed.Interdimensional coupling error is one of the main factors affecting the measurement precision of the multi-dimensional force sensors.Therefore,reducing or even eliminating dimensional coupling error becomes a fundamental requirement in the design of multi-dimensional force sensors,and the decoupling state-of-art of the multi-dimensional force sensors are introduced in this paper.At last,the trends and current challenges of multi-dimensional force sensing technology are proposed.

Real Time and Efficient Haptic Feedback Framework for Deformable Objects

On account of the high computational speed required for force feedback and the detailed graphic required for deformation feedback, this paper presents a real time and efficient haptic feedback framework by subdividing the area of interest on a relatively coarse triangular mesh. When a contact is detected, a portion of the colliding surface is subdivided and a particle system produced, which is used for deformation based on the approximate velocity cone theory. Simultaneously, a spring\|damper model is used ...

Quality-of-Experience in Human-in-the-Loop Haptic Communications

With the worldwide rapid development of 5 G networks, haptic communications, a key use case of the 5 G, has attracted increasing attentions nowadays. Its human-in-the-loop nature makes quality of experience(QoE) the leading performance indicator of the system design. A vast number of high quality works were published on user-level, application-level and network-level QoE-oriented designs in haptic communications. In this paper, we present an overview of the recent research activities in this progressive research area. We start from the QoE modeling of human haptic perceptions, followed by the application-level QoE management mechanisms based on these QoE models. High fidelity haptic communications require an orchestra of QoE designs in the application level and the quality of service(QoS) support in the network level. Hence, we also review the state-of-the-art QoS-related QoE management strategies in haptic communications, especially the QoS-related QoE modeling which guides the resource allocation design of the communication network. In addition to a thorough survey of the literature, we also present the open challenges in this research area. We believe that our review and findings in this paper not only provide a timely summary of prevailing research in this area, but also help to inspire new QoE-related research opportunities in haptic communications.