Data fusion methods in multimodal human computer dialog

In multimodal human computer dialog,non-verbal channels,such as facial expression,posture,gesture,etc,combined with spoken information,are also important in the procedure of dialogue.Nowadays,in spite of high performance of users*single channel behavior computing,it is still great challenge to understand users1 intention accurately from their multimodal behaviors.One reason for this challenge is that we still need to improve multimodal information fusion in theories,methodologies and practical systems.This paper presents a review of data fusion methods in multimodal human computer dialog.We first introduce the cognitive assumption of single channel processing,and then discuss its implementation methods in human computer dialog;for the task of multi-modal information fusion,serval computing models are presented after we introduce the principle description of multiple data fusion.Finally,some practical examples of multimodal information fusion methods are introduced and the possible and important breakthroughs of the data fusion methods in future multimodal human-computer interaction applications are discussed.

Cortical Organization of Centrifugal Afferents to the Olfactory Bulb: Mono-and Trans-synaptic Tracing with Recombinant Neurotropic Viral Tracers

Sensory processing is strongly modulated by different brain and behavioral states,and this is based on the top-down modulation.In the olfactory system,local neural circuits in the olfactory bulb(OB)are innervated by centrifugal afferents in order to regulate the processing of olfactory information in the OB under different behavioral states.The purpose of the present study was to explore the organization of neural networks in olfactory-related cortices and modulatory nuclei that give rise to direct and indirect innervations to the glomerular layer(GL)of the OB at the whole-brain scale.Injection of different recombinant attenuated neurotropic viruses into the GL showed that it received direct inputs from each layer in the OB,centrifugal inputs from the ipsilateralanterior olfactory nucleus(AON),anterior piriform cortex(Pir),and horizontal limb of diagonal band of Broca(HDB),and various indirect inputs from bilateral cortical neurons in the AON,Pir,amygdala,entorhinal cortex,hippocampus,HDB,dorsal raphe,median raphe and locus coeruleus.These results provide a circuitry basis that will help further understand the mechanism by which olfactory informationprocessing in the OB is regulated.

Neural Correlates of Spatial Navigation in Primate Hippocampus

The hippocampus has been extensively implicated in spatial navigation in rodents and more recently in bats.Numerous studies have revealed that various kinds of spatial information are encoded across hippocampal regions.In contrast,investigations of spatial behavioral correlates in the primate hippocampus are scarce and have been mostly limited to head-restrained subjects during virtual navigation.However,recent advances made in freely-moving primates suggest marked differences in spatial representations from rodents,albeit some similarities.Here,we review empirical studies examining the neural correlates of spatial navigation in the primate(including human)hippocampus at the levels of local field potentials and single units.The lower frequency theta oscillations are often intermittent.Single neuron responses are highly mixed and task-dependent.We also discuss neuronal selectivity in the eye and head coordinates.Finally,we propose that future studies should focus on investigating both intrinsic and extrinsic population activity and examining spatial coding properties in large-scale hippocampal-neocortical networks across tasks.

Next-Generation Prosthetic Hand:from Biomimetic to Biorealistic

Integrating a prosthetic hand to amputees with seamless neural compatibility presents a grand challenge to neuroscientists and neural engineers for more than half century.Mimicking anatomical structure or appearance of human hand does not lead to improved neural connectivity to the sensorimotor system of amputees.The functions of modern prosthetic hands do not match the dexterity of human hand due primarily to lack of sensory awareness and compliant actuation.Lately,progress in restoring sensory feedback has marked a significant step forward in improving neural continuity of sensory information from prosthetic hands to amputees.However,little effort has been made to replicate the compliant property of biological muscle when actuating prosthetic hands.Furthermore,a full-fledged biorealistic approach to designing prosthetic hands has not been contemplated in neuroprosthetic research.In this perspective article,we advance a novel view that a prosthetic hand can be integrated harmoniously with amputees only if neural compatibility to the sensorimotor system is achieved.Our ongoing research supports that the next-generation prosthetic hand must incorporate biologically realistic actuation,sensing,and reflex functions in order to fully attain neural compatibility.