To boost research into cognition-level visual understanding,i.e.,making an accurate inference based on a thorough understanding of visual details,visual commonsense reasoning(VCR)has been proposed.Compared with tradit...To boost research into cognition-level visual understanding,i.e.,making an accurate inference based on a thorough understanding of visual details,visual commonsense reasoning(VCR)has been proposed.Compared with traditional visual question answering which requires models to select correct answers,VCR requires models to select not only the correct answers,but also the correct rationales.Recent research into human cognition has indicated that brain function or cognition can be considered as a global and dynamic integration of local neuron connectivity,which is helpful in solving specific cognition tasks.Inspired by this idea,we propose a directional connective network to achieve VCR by dynamically reorganizing the visual neuron connectivity that is contextualized using the meaning of questions and answers and leveraging the directional information to enhance the reasoning ability.Specifically,we first develop a GraphVLAD module to capture visual neuron connectivity to fully model visual content correlations.Then,a contextualization process is proposed to fuse sentence representations with visual neuron representations.Finally,based on the output of contextualized connectivity,we propose directional connectivity to infer answers and rationales,which includes a ReasonVLAD module.Experimental results on the VCR dataset and visualization analysis demonstrate the effectiveness of our method.展开更多
AIM： To evaluate the differences in the functional connectivity（FC） of the primary visual cortex（V1） between the youth comitant exotropia（CE） patients and health subjects using resting functional magnetic reson...AIM： To evaluate the differences in the functional connectivity（FC） of the primary visual cortex（V1） between the youth comitant exotropia（CE） patients and health subjects using resting functional magnetic resonance imaging（f MRI） data.METHODS： Totally, 32 CEs（25 males and 7 females） and 32 healthy control subjects（HCs）（25 males and 7 females） were enrolled in the study and underwent the MRI scanning. Two-sample t-test was used to examine differences in FC maps between the CE patients and HCs. RESULTS： The CE patients showed significantly less FC between the left brodmann area（BA17） and left lingual gyrus/cerebellum posterior lobe, right middle occipital gyrus, left precentral gyrus/postcentral gyrus and right inferior parietal lobule/postcentral gyrus. Meanwhile, CE patients showed significantly less FC between right BA17 and right middle occipital gyrus（BA19, 37）.CONCLUSION： Our findings show that CE involves abnormal FC in primary visual cortex in many regions, which may underlie the pathologic mechanism of impaired fusion and stereoscopic vision in CEs.展开更多
基金Project supported by the National Natural Science Foundation of China(Nos.61876130 and 61932009)。
文摘To boost research into cognition-level visual understanding,i.e.,making an accurate inference based on a thorough understanding of visual details,visual commonsense reasoning(VCR)has been proposed.Compared with traditional visual question answering which requires models to select correct answers,VCR requires models to select not only the correct answers,but also the correct rationales.Recent research into human cognition has indicated that brain function or cognition can be considered as a global and dynamic integration of local neuron connectivity,which is helpful in solving specific cognition tasks.Inspired by this idea,we propose a directional connective network to achieve VCR by dynamically reorganizing the visual neuron connectivity that is contextualized using the meaning of questions and answers and leveraging the directional information to enhance the reasoning ability.Specifically,we first develop a GraphVLAD module to capture visual neuron connectivity to fully model visual content correlations.Then,a contextualization process is proposed to fuse sentence representations with visual neuron representations.Finally,based on the output of contextualized connectivity,we propose directional connectivity to infer answers and rationales,which includes a ReasonVLAD module.Experimental results on the VCR dataset and visualization analysis demonstrate the effectiveness of our method.
基金Supported by the National Natural Science Foundation of China(No.81660158No.81160118No.81400372)
文摘AIM： To evaluate the differences in the functional connectivity（FC） of the primary visual cortex（V1） between the youth comitant exotropia（CE） patients and health subjects using resting functional magnetic resonance imaging（f MRI） data.METHODS： Totally, 32 CEs（25 males and 7 females） and 32 healthy control subjects（HCs）（25 males and 7 females） were enrolled in the study and underwent the MRI scanning. Two-sample t-test was used to examine differences in FC maps between the CE patients and HCs. RESULTS： The CE patients showed significantly less FC between the left brodmann area（BA17） and left lingual gyrus/cerebellum posterior lobe, right middle occipital gyrus, left precentral gyrus/postcentral gyrus and right inferior parietal lobule/postcentral gyrus. Meanwhile, CE patients showed significantly less FC between right BA17 and right middle occipital gyrus（BA19, 37）.CONCLUSION： Our findings show that CE involves abnormal FC in primary visual cortex in many regions, which may underlie the pathologic mechanism of impaired fusion and stereoscopic vision in CEs.
