Multisensory mechanisms of gait and balance in Parkinson’s disease:an integrative review

Understanding the neural underpinning of human gait and balance is one of the most pertinent challenges for 21st-century translational neuroscience due to the profound impact that falls and mobility disturbances have on our aging population.Posture and gait control does not happen automatically,as previously believed,but rather requires continuous involvement of central nervous mechanisms.To effectively exert control over the body,the brain must integrate multiple streams of sensory information,including visual,vestibular,and somatosensory signals.The mechanisms which underpin the integration of these multisensory signals are the principal topic of the present work.Existing multisensory integration theories focus on how failure of cognitive processes thought to be involved in multisensory integration leads to falls in older adults.Insufficient emphasis,however,has been placed on specific contributions of individual sensory modalities to multisensory integration processes and cross-modal interactions that occur between the sensory modalities in relation to gait and balance.In the present work,we review the contributions of somatosensory,visual,and vestibular modalities,along with their multisensory intersections to gait and balance in older adults and patients with Parkinson’s disease.We also review evidence of vestibular contributions to multisensory temporal binding windows,previously shown to be highly pertinent to fall risk in older adults.Lastly,we relate multisensory vestibular mechanisms to potential neural substrates,both at the level of neurobiology(concerning positron emission tomography imaging)and at the level of electrophysiology(concerning electroencephalography).We hope that this integrative review,drawing influence across multiple subdisciplines of neuroscience,paves the way for novel research directions and therapeutic neuromodulatory approaches,to improve the lives of older adults and patients with neurodegenerative diseases.

DESIGN OF A NOVEL DEXTEROUS ROBOT HAND

In the robotic community more and more hands are developed. Based on theexperience of HIT Hand and DLR Hand II, a smaller and easier manufactured dexterous robot hand withmultisen-sory function and high integration is jointly developed. The prototype of the hand issuccessfully built. It has 4 fingers in total 13-DOFs (degree of freedom). Each finger has 3-DOFsand 4 joints, the last 2 joints are mechanically coupled by means of four-bar linkage mechanism. Italso has an additional DOF to realize motion of the thumb relative to the palm. The fingertip forcecan reach up to 10 N. Full integration of mechanical body, actuation system, multisensory system andelectronics is a significant feature. DSP based control system is implemented in PCI busarchitecture and the serial communication between the hand and DSP needs only 2 lines.

The use of mirror mechanism for activity-dependent neurorehabilitation

The present study observed the effects of the non-invasive, mirror neuron system application on neurorehabilitation. In primate studies, mirror neurons have been shown to fire when the subject observes or performs a specific action, thereby allowing for observation of motor cortex activation. This activation of the mirror neuron system could serve as a treatment for stroke patients. In the present study, the combination of a mirror neuron system-based therapy was introduced for the treatment of patients with motor-deficits, who could not perform rehabilitation exercises. The results also indicate that this therapeutic method plays a positive role in emotional regulation in the same patients.

Emotion Based Signal Enhancement Through Multisensory Integration Using Machine Learning

Progress in understanding multisensory integration in human have suggested researchers that the integration may result into the enhancement or depression of incoming signals.It is evident based on different psychological and behavioral experiments that stimuli coming from different perceptual modalities at the same time or from the same place,the signal having more strength under the influence of emotions effects the response accordingly.Current research inmultisensory integration has not studied the effect of emotions despite its significance and natural influence in multisensory enhancement or depression.Therefore,there is a need to integrate the emotional state of the agent with incoming stimuli for signal enhancement or depression.In this study,two different neural network-based learning algorithms have been employed to learn the impact of emotions on signal enhancement or depression.It was observed that the performance of a proposed system for multisensory integration increases when emotion features were present during enhancement or depression of multisensory signals.

Testing Multisensory Stimuli to Drive Thermal Comfort and Space Perception using VR/AR:A Design Strategy to Achieve Energy Saving

Energy consumption of the EU has a crucial environmental impact;several efforts are nowadays thus directed into massively reducing energy consumption by envelope improvement,system efficiency and smart control.On the other hand,the indoor thermal and lighting conditions significantly influence users’wellbeing and productivity,which is especially important when dealing with educational and working facilities.Strategies to enhance system efficiency are focused on design and construction aspects.These strategies ease to promote a powerful approach which is needed when focusing on existing buildings in need of retrofit measures.When dealing with new or refurbished buildings,energy saving has a further step to achieve.In the last years,research trends moved towards an increasing inclusion of human factors in energy evaluation.This allows to account for the occupancy variability in the energy analyses,considering how to bridge the performance gap between predictive models and actual consumptions due to indoor thermal settings.In empty buildings energy consumption is huge and economic efforts are wasted,due to unconscious energy-wasting behaviors.Previous studies with Interactive Virtual Environments confirmed that indoor environments illuminated by different color lights lead to the perception of different levels of thermal comfort.The results of the present paper replicate previous evidence collected in real conditions,suggesting that Virtual Reality is a valid and reliable tool to assess thermal comfort more quickly and cheaply.This paper provides a further perspective on this topic,including also the use of different fragrances to understand how the indoor environment could be enhanced and manipulated to increase wellbeing,thermal perception and finally energy saving.

AB054.Audio-visual multiple object tracking

Background:The ability to track objects as they move is critical for successful interaction with objects in the world.The multiple object tracking(MOT)paradigm has demonstrated that,within limits,our visual attention capacity allows us to track multiple moving objects among distracters.Very little is known about dynamic auditory attention and the role of multisensory binding in attentional tracking.Here,we examined whether dynamic sounds congruent with visual targets could facilitate tracking in a 3D-MOT task.Methods:Participants tracked one or multiple target-spheres among identical distractor-spheres during 8 seconds of movement in a virtual cube.In the visual condition,targets were identified with a brief colour change,but were then indistinguishable from the distractors during the movement.In the audio-visual condition,the target-spheres were accompanied by a sound,which moved congruently with the change in the target’s position.Sound amplitude varied with distance from the observer and inter-aural amplitude difference varied with azimuth.Results:Results with one target showed that performance was better in the audiovisual condition,which suggests that congruent sounds can facilitate attentional visual tracking.However,with multiple targets,the sounds did not facilitate tracking.Conclusions:This suggests that audiovisual binding may not be possible when attention is divided between multiple targets.

AB055.Eye movements in the dark:saccades to non-visual targets

Background:Saccades are rapid and abrupt eye movements that allow us to change the point of fixation very quickly.Saccades are generally made to visual points of interest,but we can also saccade to non-visual objects that attract our attention.While there is a plethora of studies investigating saccadic eye movements to visual targets,there is very little evidence of how eye movement planning occurs when individuals are performing eye movements to non-visual targets across different sensory modalities.Methods:Fifteen adults with normal,or corrected to normal,vision made saccades to either visual,auditory,tactile or proprioceptive targets.In the auditory condition a speaker was positioned at one of eight locations along a circle surrounding a central fixation point.In the proprioceptive condition the participant’s finger was placed at one of the eight locations.In the tactile condition participants were touched on their right forearm in one of four eccentric location,left and right of a central point.Eye movements were made in complete darkness.Results:We compared the precision and accuracy of the eye movements to tactile,proprioceptive,and auditory targets in the dark.Overall,both precision and accuracy of movements to non-visual targets were significantly lower compared to visual targets.Conclusions:These differences emphasize the central role of the visual system in saccade planning.

AB056. Multisensory stochastic facilitation: effect of thresholds and reaction times

Background:The concept of stochastic facilitation suggests that the addition of precise amounts of white noise can improve the perceptibility of a stimulus of weak amplitude.We know from previous research that tactile and auditory noise can facilitate visual perception,respectively.Here we wanted to see if the effects of stochastic facilitation generalise to a reaction time paradigm,and if reaction times are correlated with tactile thresholds.We know that when multiple sensory systems are stimulated simultaneously,reaction times are faster than either stimulus alone,and also faster than the sum of reaction times(known as the race model).Methods:Five participants were re-tested in five blocks each of which contained a different background noise levels,randomly ordered across sessions.At each noise level,they performed a tactile threshold detection task and a tactile reaction time task.Results:Both tactile threshold and tactile reaction times were significantly affected by the background white noise.While the preferred amplitude for the white noise was different for every participant,the average lowest threshold was obtained with white noise presented binaurally at 70 db.The reaction times were analysed by fitting an ex-Gaussian,the sum of a Gaussian function and an exponential decay function.The white noise significantly affected the exponential parameter(tau)in a way that is compatible with the facilitation of thresholds.Conclusions:We therefore conclude that multisensory reaction time facilitation can,at least in part,be explained by stochastic facilitation of the neural signals.

Multisensory Exercise Improves Balance in People with Balance Disorders: A Systematic Review

Objective To examine the effect of multisensory exercise on balance disorders.Methods PubMed,Scopus and Web of Science were searched to identify eligible studies published before January 1,2020.Eligible studies included randomized control trials(RCTs),non-randomized studies,case-control studies,and cohort studies.The methodological quality of the included studies was evaluated using JBI Critical Appraisal Checklists for RCTs and for Quasi-Experimental Studies by two researchers independently.A narrative synthesis of intervention characteristics and health-related outcomes was performed.Results A total of 11 non-randomized studies and 9 RCTs were eligible,including 667 participants.The results supported our assumption that multisensory exercise improved balance in people with balance disorders.All of the 20 studies were believed to be of high or moderate quality.Conclusion Our study confirmed that multisensory exercise was effective in improving balance in people with balance disorders.Multisensory exercises could lower the risk of fall and enhance confidence level to improve the quality of life.Further research is needed to investigate the optimal strategy of multisensory exercises and explore the underlying neural and molecular mechanisms of balance improvement brought by multisensory exercises.

Marine turtle hatchlings use multiple sensory cues to orient their crawling towards the sea:biological and conservation policy implications

The ability of sea turtle hatchlings to find the seashore soon after hatching is thought to be exclusively dependent upon visual information. Target-oriented movements in most vertebrates, however, relay on combining information gathered through different sensory systems. Hence, in this work, we investigated whether olfactory and/or magnetic information might complement visual cues during hatchling’s seaward crawling. Acute olfactory deprivation and distorted magnetic sensation in visually competent hatchlings resulted in a scattering of seaward crawling routes among cardinal points, some of them being different from those strongly preferred by control hatchlings. In addition, blindfolded hatchlings also displayed a striking misrouting while crawling on the beach surface in spite of having intact olfactory and magnetic senses. Together these results support the notion that visual information is crucial for seaward crawling, but also that olfactory and magnetic information complement visual cues when turtle hatchlings display this behavior. Hence, the present observations suggest that multisensory cues are used by turtle hatchlings while crawling towards the sea. This work also has important implications on the design of species conservation measures and policies. In the near future, efforts must be made to identify and preserve the local natural sources of odors and magnetic cues, in addition to preventing the perturbing effects of artificial lighting on adult and hatchling turtle crawling behavior.

Psychophysics of wearable haptic/tactile perception in a multisensory context

Multisensory lab based in Peking University,has carried out basic studies in multisensory space and time processing,intersensory binding and haptic/tactile perception.We exploited a typical paradigm of multisensory illusion-temporal ventriloquist effect and applied it in a wide range of multisensory interactions(mainly focused on temporal processing).In this work,we summarized how the tactile stimuli were exploited to compose tactile cues and as tactile apparent motion to interface with other sensory stimuli(visual and auditory stimuli)to examine the underlying perceptual organization in a multisensory context.Moreover,we introduced two examples of wearable haptic/tactile perception in our lab,by using two customized tactile devices and discussed the potential applications in this field.

Cortical Mechanisms of Multisensory Linear Self-motion Perception

Accurate self-motion perception,which is critical for organisms to survive,is a process involving multiple sensory cues.The two most powerful cues are visual(optic flow)and vestibular(inertial motion).Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction,often in a statistically Bayesian-optimal way.In the last decade,single-unit recordings in awake,behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution,giving insight into possible neural mechanisms underlying multisensory self-motion perception.Here,we review these findings,along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities.We show that,in light of new data,conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged.We propose that different temporal component signals may mediate different functions,a possibility that requires future studies.

Simulating tactile and visual multisensory behaviour in humans based on an MoS_(2) field effect transistor

A visual and tactile multisensory integrated system is essential for human walking due to the demand for real-time interactions between perception and action.Here,a piezoresistor and MoS_(2)field effect transistor are combined to construct an artificial integration nervous system to simulate perception and synaptic plasticity.The key characteristics of synaptic plasticity are successfully demonstrated by individual pressure signals,individual optical signals,the synergy of optical and pressure signals,which are based on the electron trapping–detrapping mechanism at the MoS_(2)/SiO_(2)interface.We demonstrate that perception under synergy is stronger than perception under optical or pressure signal alone,which is similar to a biological system.Moreover,various distinguishable motion scenarios(combination of the following conditions:external lighting environment of day or night,flat or rough road,movement state of walking or running)are simulated and verified by adjusting the amplitude and frequency of the optical and pressure signals.

Temperature-controlled multisensory neuromorphic devices for artificial visual dynamic capture enhancement

Multi-sensory neuromorphic devices(MND)have broad potential in overcoming the structural bottleneck of von Neumann in the era of big data.However,the current multisensory artificial neuromorphic system is mainly based on unitary nonvolatile memory or volatile synaptic devices without intrinsic thermal sensitivity,which limits the range of biological multisensory perception and the flexibility and computational efficiency of the neural morphological computing system.Here,a temperature-dependent memory/synaptic hybrid artificial neuromorphic device based on floating gate phototransistors(FGT)is fabricated.The CsPbBr_(3)/TiO_(2)core–shell nanocrystals(NCs)prepared by in-situ pre-protection low-temperature solvothermal method were used as the photosensitive layer.The device exhibits remarkable multi-level visual memory with a large memory window of 59.6 V at room temperature.Surprisingly,when the temperature varies from 20 to 120℃back and forth,the device can switch between nonvolatile memory and volatile synaptic device with reconfigurable and reversible behaviors,which contributes to the efficient visual/thermal fusion perception.This work expands the sensory range of multisensory devices and promotes the development of memory and neuromorphic devices based on organic field-effect transistors(OFET).

Alteration of the temporal association between courtship audio and visual components affects female sexual response

Some multimodal signals-that is,occurring in more than one sensory modality-appear to carry additional information which is not present when component signals are presented separately.To understand the function of male ring dove’s(Streptopelia risoria)multimodal courtship,we used audiovisual playback of male displays to investigate female response to stimuli differing in their audiovisual timing.From natural courtship recordings,we created a shifted stimulus where audio was shifted relative to video by afixed value and a jittered stimulus,where each call was moved randomly along the visual channel.We presented 3 groups of females with the same stimulus type,that is,control,shifted,and jittered,for 7 days.We recorded their behavior and assessed pre-and post-test blood estradiol concentration.We found that playback exposure increased estradiol levels,confirming that this technique can be efficiently used to study doves’sexual communication.Additionally,chasing behavior(indicating sexual stimulation)increased over experimental days only in the control condition,suggesting a role of multimodal timing on female response.This stresses the importance of signal configuration in multimodal communication,as additional information is likely to be contained in the temporal association between modalities.

Virtual reality systems for rodents

Over the last decade virtual reality （VR） setups for rodents have been developed and utilized to in-vestigate the neural foundations of behavior. Such VR systems became very popular since theyallow the use of state-of-the-art techniques to measure neural activity in behaving rodents that can-not be easily used with classical behavior setups. Here, we provide an overview of rodent VR tech-nologies and review recent results from related research. We discuss commonalities and differ-ences as well as merits and issues of different approaches. A special focus is given to experimental（behavioral） paradigms in use. Finally we comment on possible use cases that may further exploitthe potential of VR in rodent research and hence inspire future studies.

Spatio-temporal measures of electrophysiological correlates for behavioral multisensory enhancement during visual,auditory and somatosensory stimulation:A behavioral and ERP study

Multisensory enhancement,as a facilitation phenomenon,is responsible for superior behavioral performance when an individual is responding to cross-modal versus modality-specific stimuli.However,the event-related potential（ERP） counterparts of behavioral multisensory enhancement are not well known.We recorded ERPs and behavioral data from 14 healthy volunteers with three types of target stimuli（modality-specific,bimodal,and trimodal） to examine the spatio-temporal electrophysiological characteristics of multisensory enhancement by comparing behavioral data with ERPs.We found a strong correlation between P3 latency and behavioral performance in terms of reaction time（RT）（R = 0.98,P ＆lt;0.001）,suggesting that P3 latency constitutes a temporal measure of behavioral multisensory enhancement.In addition,a fast RT and short P3 latency were found when comparing the modality-specific visual target with the modality-specific auditory and somatosensory targets.Our results indicate that behavioral multisensory enhancement can be identified by the latency and source distribution of the P3 component.These findings may advance our understanding of the neuronal mechanisms of multisensory enhancement.

Changes in the Vision-related Resting-state Network in Pituitary Adenoma Patients After Vision Improvement

Background：The aim of this research was to investigate the changes in the vision-related resting-state network （V-RSN） in pituitary adenoma （PA） patients after vision improvement,which was induced by operative treatment.Methods：Ten PA patients with an improved visual acuity or/and visual field after transsphenoidal pituitary tumor resection were recruited and underwent a complete neuro-ophthalmologic evaluation,as well as an magnetic resonance imaging （MRI） protocol,including structural and resting-state functional MRI sequences before and after the operation.The regional homogeneity （ReHo） of the V-RSN was evaluated.Two sample t-test was performed to identify the significant differences in the V-RSN in the PA patients before and after transsphenoidal pituitary tumor resection.Results：Compared with the preoperation counterparts,the PA patients with improved vision after the operation exhibited reduced ReHo in the bilateral thalamus,globus pallidus,caudate nucleus,putamen nucleus,supplementary motor area,and left hippocampal formation,and increased ReHo in the bilateral cuneus gyrus,calcarine gyrus,right lingual gyrus,and fusiform gyrus.Conclusions：PA patients with improved vision exhibit increased neural activity within the visual cortex,but decreased neural activity in subareas of the multisensory and multimodal systems beyond the vision cortex.

Visual Dominance Effect upon Passing the Central Bottleneck of Information Processing

Background： In the classical psychological refractory period （PRP） paradigm, two stimuli are presented in brief succession, and participants are asked to make separate speeded responses to both stimuli. Due to a central cognitive bottleneck, responses to the second stimulus are delayed, especially at short stimulus-onset asynchrony （SOA） between the two stimuli. Although the mechanisms of dual-task interference in the classical PRP paradigm have been extensively investigated, specific mechanisms underlying the cross-modal PRP paradigm are not well understood. In particular, it remains unknown whether the dominance of vision over audition manifests in the cross-modal PRP tasks. The present study aimed to investigate whether the visual dominance effect manifessts in the cross-modal PRP paradigm. Methods： We adapted the classical PRP paradigm by manipulating the order of a visual and an auditory task： the visual task could either precede the auditory task or vice versa, at either short or long SOAs. Twenty-five healthy participants took part in Experiment 1, and thirty-three new participants took part in Experiment 2. Reaction time and accuracy data were calculated and further analyzed by repeated-measures analysis of variance. Results： The results showed that visual precedence in the Visual-Auditory condition caused larger impairments to the subsequent auditory processing than vice versa in the Auditory-Visual condition： a larger delay of second response was revealed in the Visual-Auditory condition （ 135 ± 10 ms） than the Auditory-Visual condition （88 ± 9 ms）. This effect was found only at the short SOAs tinder the existence of the central bottleneck, but not at the long SOAs. Moreover, this effect occurred both when the single visual and the single auditory task were of equal difficulty in Experiment I and when the single auditory task was more difficult than the single visual task in Experiment 2. Conclusion： Results of the two experiments suggested that the visual dominance effect occurred under the central bottleneck of cognitive processing.