Prediction of cognitive scores by joint use of movie-watching fMRI connectivity and eye tracking via Attention-CensNet

Background:Brain functional connectivity under the naturalistic paradigm has been shown to be better at predicting individual behaviors than other brain states,such as rest and doing tasks.Nevertheless,the state-of-the-art methods have found it difcult to achieve desirable results from movie watching paradigm functional magnetic resonance imaging(mfMRI)-induced brain functional connectivity,especially when there are fewer datasets.Incorporating other physical measurements into the prediction method may enhance accuracy.Eye tracking,becoming popular due to its portability and lower expense,can provide abundant behavioral features related to the output of human's cognition,and thus might supplement the mfMRI in observing participants'subconscious behaviors.However,there are very few studies on how to effectively integrate the multimodal information to strengthen the performance by a unified framework.objective:A fusion approach with mfMRI and eye tracking,based on convolution with edge-node switching in graph neural networks(CensNet),is proposed in this article.Methods:In this graph model,participants are designated as nodes,mfMRI derived functional connectivity as node features,and different eye-tracking features are used to compute similarity between participants to construct heterogeneous graph edges.By taking multiple graphs as different channels,we introduce squeeze-and-excitation attention module to CensNet(A-CensNet)to integrate graph embeddings from multiple channels into one.Results:The proposed model outperforms those using a single modality and single channel,and state-of-the-art methods.Conclusions:The results indicate that brain functional activities and eye behaviors might complement each other in interpreting trait-like phenotypes.

Fundamental functional differences between gyri and sulci: implications for brain function, cognition, and behavior

Folding of the cerebral cortex is a prominent characteristic of mammalian brains.Alterations or deficits in cortical folding are strongly correlated with abnormal brain function,cognition,and behavior.Therefore,a precise mapping between the anatomy and function of the brain is critical to our understanding of the mechanisms of brain structural architecture in both health and diseases.Gyri and sulci,the standard nomenclature for cortical anatomy,serve as building blocks to make up complex folding patterns,providing a window to decipher cortical anatomy and its relation with brain functions.Huge efforts have been devoted to this research topic from a variety of disciplines including genetics,cell biology,anatomy,neuroimaging,and neurology,as well as involving computational approaches based on machine learning and artificial intelligence algorithms.However,despite increasing progress,our understanding of the functional anatomy of gyro-sulcal patterns is still in its infancy.In this review,we present the current state of this field and provide our perspectives of the methodologies and conclusions concerning functional differentiation between gyri and sulci,as well as the supporting information from genetic,cell biology,and brain structure research.In particular,we will further present a proposed framework for attempting to interpret the dynamic mechanisms of the functional interplay between gyri and sulci.Hopefully,this review will provide a comprehensive summary of anatomo-functional relationships in the cortical gyro-sulcal system together with a consideration of how these contribute to brain function,cognition,and behavior,as well as to mental disorders.