Eye-shaped keyboard for dual-hand text entry in virtual reality

We propose an eye-shaped keyboard for high-speed text entry in virtual reality (VR), having the shape of dual eyes with characters arranged along the curved eyelids, which ensures low density and short spacing of the keys. The eye-shaped keyboard references the QWERTY key sequence, allowing the users to benefit from their experience using the QWERTY keyboard. The user interacts with an eye-shaped keyboard using rays controlled with both the hands. A character can be entered in one step by moving the rays from the inner eye regions to regions of the characters. A high-speed auto-complete system was designed for the eye-shaped keyboard. We conducted a pilot study to determine the optimal parameters, and a user study to compare our eye-shaped keyboard with the QWERTY and circular keyboards. For beginners, the eye-shaped keyboard performed significantly more efficiently and accurately with less task load and hand movement than the circular keyboard. Compared with the QWERTY keyboard, the eye-shaped keyboard is more accurate and significantly reduces hand translation while maintaining similar efficiency. Finally, to evaluate the potential of eye-shaped keyboards, we conducted another user study. In this study, the participants were asked to type continuously for three days using the proposed eye-shaped keyboard, with two sessions per day. In each session, participants were asked to type for 20min, and then their typing performance was tested. The eye-shaped keyboard was proven to be efficient and promising, with an average speed of 19.89 words per minute (WPM) and mean uncorrected error rate of 1.939%. The maximum speed reached 24.97 WPM after six sessions and continued to increase.

Haptic display for virtual reality: progress and challenges

Immersion, interaction, and imagination are three features of virtual reality (VR). Existing VR systems possess fairly realistic visual and auditory feedbacks, and however, are poor with haptic feedback, by means of which human can perceive the physical world via abundant haptic properties. Haptic display is an interface aiming to enable bilateral signal communications between human and computer, and thus to greatly enhance the immersion and interaction of VR systems. This paper surveys the paradigm shift of haptic display occurred in the past 30 years, which is classified into three stages, including desktop haptics, surface haptics, and wearable haptics. The driving forces, key technologies and typical applications in each stage are critically reviewed. Toward the future high-fidelity VR interaction, research challenges are highlighted concerning handheld haptic device, multimodal haptic device, and high fidelity haptic rendering. In the end, the importance of understanding human haptic perception for designing effective haptic devices is addressed.

Attention-Based Deep Learning Model for Image Desaturation of SDO/AIA

The Atmospheric Imaging Assembly(AIA)onboard the Solar Dynamics Observatory(SDO)captures full-disk solar images in seven extreme ultraviolet wave bands.As a violent solar flare occurs,incoming photoflux may exceed the threshold of an optical imaging system,resulting in regional saturation/overexposure of images.Fortunately,the lost signal can be partially retrieved from non-local unsaturated regions of an image according to scattering and diffraction principle,which is well consistent with the attention mechanism in deep learning.Thus,an attention augmented convolutional neural network(AANet)is proposed to perform image desaturation of SDO/AIA in this paper.It is built on a U-Net backbone network with partial convolution and adversarial learning.In addition,a lightweight attention model,namely criss-cross attention,is embedded between each two convolution layers to enhance the backbone network.Experimental results validate the superiority of the proposed AANet beyond state-of-the-arts from both quantitative and qualitative comparisons.

Virtual reality and augmented reality in medical simulation

Virtual reality/augmented reality(VR/AR)technologies have been widely used in medical fields,such as surgical simulation,healthcare,surgical navigation,computer assisted diagnosis,medical visualization and translational medicine.In this special issue,we include 6 research papers on the application of VR/AR to medical simulation,which can be divided into two sets,including surgical simulation and medical modelling/analysis.

Haptic feedback for virtual reality

Touch is an important channel for human beings to communicate with the external world and plays a crucial role in human interaction with nature.Information such as softness,friction,texture and warmth,or more complicated human emotional communication can only be perceived by the act of touching.Despite its importance,it is striking that haptic feeback in human-machine interaction is still in its infancy.

Hawk‐eye‐inspired perception algorithm of stereo vision for obtaining orchard 3D point cloud navigation map

The binocular stereo vision is the lowest cost sensor for obtaining 3D information.Considering the weakness of long‐distance measurement and stability,the improvement of accuracy and stability of stereo vision is urgently required for application of precision agriculture.To address the challenges of stereo vision long‐distance measurement and stable perception without hardware upgrade,inspired by hawk eyes,higher resolution perception and the adaptive HDR(High Dynamic Range)were introduced in this paper.Simulating the function from physiological structure of‘deep fovea’and‘shallow fovea’of hawk eye,the higher resolution reconstruction method in this paper was aimed at ac-curacy improving.Inspired by adjustment of pupils,the adaptive HDR method was proposed for high dynamic range optimisation and stable perception.In various light conditions,compared with default stereo vision,the accuracy of proposed algorithm was improved by 28.0%evaluated by error ratio,and the stability was improved by 26.56%by disparity accuracy.For fixed distance measurement,the maximum improvement was 78.6%by standard deviation.Based on the hawk‐eye‐inspired perception algorithm,the point cloud of orchard was improved both in quality and quantity.The hawk‐eye‐inspired perception algorithm contributed great advance in binocular 3D point cloud recon-struction in orchard navigation map.

Effects of virtual environment and self-representations on perception and physical performance in redirected jumping

Background Redirected jumping(RDJ)allows users to explore virtual environments(VEs)naturally by scaling a small real-world jump to a larger virtual jump with virtual camera motion manipulation,thereby addressing the problem of limited physical space in VR applications.Previous RDJ studies have mainly focused on detection threshold estimation.However,the effect VE or self representation(SR)has on the perception or performance of RDJs remains unclear.Methods In this paper,we report experiments to measure the perception(detection thresholds for gains,presence,embodiment,intrinsic motivation,and cybersickness)and physical performance(heart rate intensity,preparation time,and actual jumping distance)of redirected forward jumping under six different combinations of VE(low and high visual richness)and SRs(invisible,shoes,and human-like).Results Our results indicated that the detection threshold ranges for horizontal translation gains were significantly smaller in the VE with high rather than low visual richness.When different SRs were applied,our results did not suggest significant differences in detection thresholds,but it did report longer actual jumping distances in the invisible body case compared with the other two SRs.In the high visual richness VE,the preparation time for jumping with a human-like avatar was significantly longer than that with other SRs.Finally,some correlations were found between perception and physical performance measures.Conclusions All these findings suggest that both VE and SRs influence users'perception and performance in RDJ and must be considered when designing locomotion techniques.

Web-based mixed reality video fusion with remote rendering

Background Mixed reality(MR)video fusion systems merge video imagery with 3D scenes to make the scene more realistic and help users understand the video content and temporal–spatial correlation between them,reducing the user′s cognitive load.MR video fusion are used in various applications;however,video fusion systems require powerful client machines because video streaming delivery,stitching,and rendering are computationally intensive.Moreover,huge bandwidth usage is another critical factor that affects the scalability of video-fusion systems.Methods Our framework proposes a fusion method for dynamically projecting video images into 3D models as textures.Results Several experiments on different metrics demonstrate the effectiveness of the proposed framework.Conclusions The framework proposed in this study can overcome client limitations by utilizing remote rendering.Furthermore,the framework we built is based on browsers.Therefore,the user can test the MR video fusion system with a laptop or tablet without installing any additional plug-ins or application programs.

Prescribed Performance Evolution Control for Quadrotor Autonomous Shipboard Landing

The shipboard landing problem for a quadrotor is addressed in this paper,where the ship trajectory tracking control issue is transformed into a stabilization control issue by building a relative position model.To guarantee both transient performance and steady-state landing error,a prescribed performance evolution control(PPEC)method is developed for the relative position control.In addition,a novel compensation system is proposed to expand the performance boundaries when the input saturation occurs and the error exceeds the predefined threshold.Considering the wind and wave on the relative position model,an adaptive sliding mode observer(ASMO)is designed for the disturbance with unknown upper bound.Based on the dynamic surface control framework,a shipboard landing controller integrating PPEC and ASMO is established for the quadrotor,and the relative position control error is guaranteed to be uniformly ultimately bounded.Simulation results have verified the feasibility and effectiveness of the proposed shipboard landing control scheme.

Development of a Hand Exoskeleton System for Index Finger Rehabilitation

In order to overcome the drawbacks of traditional rehabilitation method,the robot-aided rehabilitation has been widely investigated for the recent years.And the hand rehabilitation robot,as one of the hot research fields,remains many challenging issues to be investigated.This paper presents a new hand exoskeleton system with some novel characteristics.Firstly,both active and passive rehabilitative motions are realized.Secondly,the device is elaborately designed and brings advantages in many aspects.For example,joint motion is accomplished by a parallelogram mechanism and high level motion control is therefore made very simple without the need of complicated kinematics.The adjustable joint limit design ensures that the actual joint angles don＇t exceed the joint range of motion（ROM） and thus the patient safety is guaranteed.This design can fit to the different patients with different joint ROM as well as to the dynamically changing ROM for individual patient.The device can also accommodate to some extent variety of hand sizes.Thirdly,the proposed control strategy simultaneously realizes the position control and force control with the motor driver which only works in force control mode.Meanwhile,the system resistance compensation is preliminary realized and the resisting force is effectively reduced.Some experiments were conducted to verify the proposed system.Experimentally collected data show that the achieved ROM is close to that of a healthy hand and the range of phalange length（ROPL） covers the size of a typical hand,satisfying the size need of regular hand rehabilitation.In order to evaluate the performance when it works as a haptic device in active mode,the equivalent moment of inertia（MOI） of the device was calculated.The results prove that the device has low inertia which is critical in order to obtain good backdrivability.The experiments also show that in the active mode the virtual interactive force is successfully feedback to the finger and the resistance is reduced by one-third;for the passive control mode,the desired trajectory is realized satisfactorily.

SIMULATION OF VIBRATION STRESS RELIEF AFTER WELDING BASED ON FEM

A finite element model is developed for the simulation of vibration stress relief （VSR） after welding. For the nonresonant vibration, the reduction in stress strongly depends on the amplitude of vibration. For the resonant vibration, the vibration frequency is the key for stress relief. The vibration frequency should be close to the structure natural frequency for the desired vibration mode. Only small vibration amplitude is required, which will be amplified during vibration. Vibration time does not have a major impact on vibration stress relief. When the amplitude of vibration stress relief is large, the treatment will be more effective.

Differently implicational α-universal triple I restriction method of (1, 2, 2) type

From the viewpoints of both fuzzy system and fuzzy reasoning, a new fuzzy reasoning method which contains the α- triple I restriction method as its particular case is proposed. The previous α-triple I restriction principles are improved, and then the optimal restriction solutions of this new method are achieved, especially for seven familiar implications. As its special case, the corresponding results of α-triple I restriction method are obtained and improved. Lastly, it is found by examples that this new method is more reasonable than the α-triple I restriction method.

Dynamic characteristic analysis of whole machine tools based on Kriging model

In order to study the variation of machine tools’dynamic characteristics in the manufacturing space,a Kriging approximate model is proposed.Finite element method(FEM)is employed on the platform of ANSYS to establish finite element(FE)model with the dynamic characteristic of combined interface for a milling machine,which is newly designed for producing aero engine blades by a certain enterprise group in China.The stiffness and damping of combined interfaces are adjusted by using adaptive simulated annealing algorithm with the optimizing software of iSIGHT in the process of FE model update according to experimental modal analysis(EMA)results.The Kriging approximate model is established according to the finite element analysis results utilizing orthogonal design samples by taking into account of the range of configuration parameters.On the basis of the Kriging approximate model,the response surfaces between key response parameter and configuration parameters are obtained.The results indicate that configuration parameters have great effects on dynamic characteristics of machine tools,and the Kriging approximate model is an effective and rapid method for estimating dynamic characteristics of machine tools in the manufacturing space.

Design of a Lightweight Force-Feedback Glove with a Large Workspace

A wearable force-feedback glove is a promising way to enhance the immersive sensation when a user interacts with virtual objects in virtual reality scenarios.Design challenges for such a glove include allowing a large fingertip workspace,providing a desired force sensation when simulating both free-and constrained-space interactions,and ensuring a lightweight structure.In this paper,we present a forcefeedback glove using a pneumatically actuated mechanism mounted on the dorsal side of the user’s hand.By means of a triple kinematic paired link with a curved sliding slot,a hybrid cam-linkage mechanism is proposed to transmit the resistance from the pneumatic piston rod to the fingertip.In order to obtain a large normal component of the feedback force on the user’s fingertip,the profile of the sliding slot was synthesized through an analysis of the force equilibrium on the triple kinematic paired link.A prototype five-fingered glove with a mass of 245 g was developed,and a wearable force-measurement system was constructed to permit the quantitative evaluation of the interaction performance in both free and constrained space.The experimental results confirm that the glove can achieve an average resistance of less than 0.1 N in free-space simulation and a maximum fingertip force of 4 N in constrained-space simulation.The experiment further confirms that this glove permits the finger to move freely to simulate typical grasping gestures.

Virtual brain surgery simulation system based on haptic interaction

To improve the accuracy and interactivity of soft tissue deformation simulation,a new plate spring model based on physics is proposed.The model is parameterized and thus can be adapted to simulate different organs.Different soft tissues are modeled by changing the width,number of pieces,thickness,and length of a single plate spring.In this paper,the structural design,calculation of soft tissue deformation and real-time feedback operations of our system are also introduced.To evaluate the feasibility of the system and validate the model,an experimental system of haptic interaction,in which users can use virtual hands to pull virtual brain tissues,is built using PHANTOM OMNI devices.Experimental results show that the proposed system is stable,accurate and promising for modeling instantaneous soft tissue deformation.

Load Sharing Behavior of Star Gearing Reducer for Geared Turbofan Engine

Load sharing behavior is very important for power-split gearing system, star gearing reducer as a new type and special transmission system can be used in many industry fields. However, there is few literature regarding the key multiple-split load sharing issue in main gearbox used in new type geared turbofan engine. Further mechanism anal- ysis are made on load sharing behavior among star gears of star gearing reducer for geared turbofan engine. Compre- hensive meshing error analysis are conducted on eccentricity error, gear thickness error, base pitch error, assembly error, and bearing error of star gearing reducer respectively. Floating meshing error resulting from meshing clearance variation caused by the simultaneous floating of sun gear and annular gear are taken into account. A refined mathematical model for load sharing coefficient calculation is established in consideration of different meshing stiffness and support- ing stiffness for components. The regular curves of load sharing coefficient under the influence of interactions, single action and single variation of various component errors are obtained. The accurate sensitivity of load sharing coefficienttoward different errors is mastered. The load sharing coef- ficient of star gearing reducer is 1.033 and the maximum meshing force in gear tooth is about 3010 N. This paper provides scientific theory evidences for optimal parameter design and proper tolerance distribution in advanced devel- opment and manufacturing process, so as to achieve optimal effects in economy and technology.

Real-time haptic model for soft tissue deformation in surgery simulation

The modelling and simulation of deformable objects is a challenging topic in the field of haptic rendering between human and virtual environment.In this paper,a novel and efficient layered rhombus-chain-connected haptic deformation model based on physics is proposed for an excellent haptic rendering.During the modelling,the accumulation of relative displacements in every chain structure unit in each layer is equal to the deformation on the virtual object surface,and the resultant force of corresponding springs is equivalent to the external force.The layered rhombus-chain-connected model is convenient and fast to calculate,and can satisfy real-time requirement due to its simple nature.Simulation experiments in virtual human liver based on the proposed model are conducted,and the results demonstrate that our model provides stable and realistic haptic feeling in real time.Meanwhile,the display result is vivid.

Super-resolution of Solar Magnetograms Using Deep Learning

Currently,data-driven models of solar activity forecast are investigated extensively by using machine learning.For model training,it is highly demanded to establish a large database which may contain observations coming from different instruments with different spatio-temporal resolutions.In this paper,we employ deep learning models for super-resolution(SR)of magnetogram of Michelson Doppler Imager(MDI)in order to achieve the same spatial resolution of Helioseismic and Magnetic Imager(HMI).First,a generative adversarial network(GAN)is designed to transfer characteristics of MDI onto downscaled HMI,getting low-resolution HMI magnetogram in the same domain as MDI.Then,with the paired low-resolution and high-resolution HMI magnetograms,another GAN is trained in a supervised learning way,which consists of two streams,one is for generating high-fidelity image content,the other is explicitly optimized for generating elaborate image gradients.Thus,these two streams work together to guarantee both high-fidelity and photorealistic super-resolved images.Experimental results demonstrate that the proposed method can generate super-resolved magnetograms with perceptual-pleasant visual quality.Meanwhile,the best PSNR,LPIPS,RMSE,comparable SSIM and CC are obtained by the proposed method.The source code and data set can be accessed via https://github.com/filterbank/SPSR.

Semantic categorization of indoor places using CNN for mobile robot exploration

The ability of achieving a semantic understanding of workspaces is an important capability for mobile robot. A method is proposed to categorize different places in a typical indoor environment by using a Kinect sensors for mobile robot exploration. At first,the invariant feature based images stitching approach is adopted to form a panoramic image according to Kinect visual information,and the translation between Kinect depth information and obstacle distance information is performed to obtain virtual LIDAR data. Then,the semantic classifier is designed by using convolutional neural networks( CNN) for indoor place categorization based on Kinect visual observations with panoramic view. At last,a frontier-based exploration method is applied to carry out indoor autonomous exploration of mobile robots,which integrates the CNN-based categorization approach. The proposed method has been implemented and tested on a real robot,and experiment results demonstrate the approach effectiveness on solving the semantic categorization problem for mobile robot exploration.

A mathematical morphological approach for region of interest coding of microscopy image compression

A novel mathematical morphological approach for region of interest(ROI) automatic determination and JPEG2000-based coding of microscopy image compression is presented.The algorithm is very fast and requires lower computing power,which is particularly suitable for some irregular region-based cell microscopy images with poor qualities.Firstly,an active threshold-based method is discussed to create a rough mask of regions of interest(cells).And then some morphological operations are designed and applied to achieve the segmentation of cells.In addition,an extra morphological operation,dilation,is applied to create the final mask with some redundancies to avoid the"edge effect"after removing false cells.Finally,ROI and region of background(ROB) are obtained and encoded individually in different compression ratio flexibly based on the JPEG2000,which can adjust the quality between ROI and ROB without coding for ROI shape.The experimental results certify the effectiveness of the proposed algorithm,and compared with JPEG2000,the proposed algorithm has better performance in both subjective quality and objective quality at the same compression ratios.