Emotion Based Signal Enhancement Through Multisensory Integration Using Machine Learning

Progress in understanding multisensory integration in human have suggested researchers that the integration may result into the enhancement or depression of incoming signals.It is evident based on different psychological and behavioral experiments that stimuli coming from different perceptual modalities at the same time or from the same place,the signal having more strength under the influence of emotions effects the response accordingly.Current research inmultisensory integration has not studied the effect of emotions despite its significance and natural influence in multisensory enhancement or depression.Therefore,there is a need to integrate the emotional state of the agent with incoming stimuli for signal enhancement or depression.In this study,two different neural network-based learning algorithms have been employed to learn the impact of emotions on signal enhancement or depression.It was observed that the performance of a proposed system for multisensory integration increases when emotion features were present during enhancement or depression of multisensory signals.

Multisensory mechanisms of gait and balance in Parkinson’s disease:an integrative review

Understanding the neural underpinning of human gait and balance is one of the most pertinent challenges for 21st-century translational neuroscience due to the profound impact that falls and mobility disturbances have on our aging population.Posture and gait control does not happen automatically,as previously believed,but rather requires continuous involvement of central nervous mechanisms.To effectively exert control over the body,the brain must integrate multiple streams of sensory information,including visual,vestibular,and somatosensory signals.The mechanisms which underpin the integration of these multisensory signals are the principal topic of the present work.Existing multisensory integration theories focus on how failure of cognitive processes thought to be involved in multisensory integration leads to falls in older adults.Insufficient emphasis,however,has been placed on specific contributions of individual sensory modalities to multisensory integration processes and cross-modal interactions that occur between the sensory modalities in relation to gait and balance.In the present work,we review the contributions of somatosensory,visual,and vestibular modalities,along with their multisensory intersections to gait and balance in older adults and patients with Parkinson’s disease.We also review evidence of vestibular contributions to multisensory temporal binding windows,previously shown to be highly pertinent to fall risk in older adults.Lastly,we relate multisensory vestibular mechanisms to potential neural substrates,both at the level of neurobiology(concerning positron emission tomography imaging)and at the level of electrophysiology(concerning electroencephalography).We hope that this integrative review,drawing influence across multiple subdisciplines of neuroscience,paves the way for novel research directions and therapeutic neuromodulatory approaches,to improve the lives of older adults and patients with neurodegenerative diseases.

Multisensory Exercise Improves Balance in People with Balance Disorders: A Systematic Review

Objective To examine the effect of multisensory exercise on balance disorders.Methods PubMed,Scopus and Web of Science were searched to identify eligible studies published before January 1,2020.Eligible studies included randomized control trials(RCTs),non-randomized studies,case-control studies,and cohort studies.The methodological quality of the included studies was evaluated using JBI Critical Appraisal Checklists for RCTs and for Quasi-Experimental Studies by two researchers independently.A narrative synthesis of intervention characteristics and health-related outcomes was performed.Results A total of 11 non-randomized studies and 9 RCTs were eligible,including 667 participants.The results supported our assumption that multisensory exercise improved balance in people with balance disorders.All of the 20 studies were believed to be of high or moderate quality.Conclusion Our study confirmed that multisensory exercise was effective in improving balance in people with balance disorders.Multisensory exercises could lower the risk of fall and enhance confidence level to improve the quality of life.Further research is needed to investigate the optimal strategy of multisensory exercises and explore the underlying neural and molecular mechanisms of balance improvement brought by multisensory exercises.

CSMCCVA:Framework of cross-modal semantic mapping based on cognitive computing of visual and auditory sensations

Cross-modal semantic mapping and cross-media retrieval are key problems of the multimedia search engine.This study analyzes the hierarchy,the functionality,and the structure in the visual and auditory sensations of cognitive system,and establishes a brain-like cross-modal semantic mapping framework based on cognitive computing of visual and auditory sensations.The mechanism of visual-auditory multisensory integration,selective attention in thalamo-cortical,emotional control in limbic system and the memory-enhancing in hippocampal were considered in the framework.Then,the algorithms of cross-modal semantic mapping were given.Experimental results show that the framework can be effectively applied to the cross-modal semantic mapping,and also provides an important significance for brain-like computing of non-von Neumann structure.

Cortical Mechanisms of Multisensory Linear Self-motion Perception

Accurate self-motion perception,which is critical for organisms to survive,is a process involving multiple sensory cues.The two most powerful cues are visual(optic flow)and vestibular(inertial motion).Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction,often in a statistically Bayesian-optimal way.In the last decade,single-unit recordings in awake,behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution,giving insight into possible neural mechanisms underlying multisensory self-motion perception.Here,we review these findings,along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities.We show that,in light of new data,conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged.We propose that different temporal component signals may mediate different functions,a possibility that requires future studies.

Marine turtle hatchlings use multiple sensory cues to orient their crawling towards the sea:biological and conservation policy implications

The ability of sea turtle hatchlings to find the seashore soon after hatching is thought to be exclusively dependent upon visual information. Target-oriented movements in most vertebrates, however, relay on combining information gathered through different sensory systems. Hence, in this work, we investigated whether olfactory and/or magnetic information might complement visual cues during hatchling’s seaward crawling. Acute olfactory deprivation and distorted magnetic sensation in visually competent hatchlings resulted in a scattering of seaward crawling routes among cardinal points, some of them being different from those strongly preferred by control hatchlings. In addition, blindfolded hatchlings also displayed a striking misrouting while crawling on the beach surface in spite of having intact olfactory and magnetic senses. Together these results support the notion that visual information is crucial for seaward crawling, but also that olfactory and magnetic information complement visual cues when turtle hatchlings display this behavior. Hence, the present observations suggest that multisensory cues are used by turtle hatchlings while crawling towards the sea. This work also has important implications on the design of species conservation measures and policies. In the near future, efforts must be made to identify and preserve the local natural sources of odors and magnetic cues, in addition to preventing the perturbing effects of artificial lighting on adult and hatchling turtle crawling behavior.