fMRI-based Decoding of Visual Information from Human Brain Activity: A Brief Review

One of the most significant challenges in the neuroscience community is to understand how the human brain works.Recent progress in neuroimaging techniques have validated that it is possible to decode a person′s thoughts,memories,and emotions via functional magnetic resonance imaging(i.e.,fMRI)since it can measure the neural activation of human brains with satisfied spatiotemporal resolutions.However,the unprecedented scale and complexity of the fMRI data have presented critical computational bottlenecks requiring new scientific analytic tools.Given the increasingly important role of machine learning in neuroscience,a great many machine learning algorithms are presented to analyze brain activities from the fMRI data.In this paper,we mainly provide a comprehensive and up-to-date review of machine learning methods for analyzing neural activities with the following three aspects,i.e.,brain image functional alignment,brain activity pattern analysis,and visual stimuli reconstruction.In addition,online resources and open research problems on brain pattern analysis are also provided for the convenience of future research.

Deep Natural Image Reconstruction from Human Brain Activity Based on Conditional Progressively Growing Generative Adversarial Networks

Brain decoding based on functional magnetic resonance imaging has recently enabled the identification of visual perception and mental states.However,due to the limitations of sample size and the lack of an effective reconstruction model,accurate reconstruction of natural images is still a major challenge.The current,rapid development of deep learning models provides the possibility of overcoming these obstacles.Here,we propose a deep learning-based framework that includes a latent feature extractor,a latent feature decoder,and a natural image generator,to achieve the accurate reconstruction of natural images from brain activity.The latent feature extractor is used to extract the latent features of natural images.The latent feature decoder predicts the latent features of natural images based on the response signals from the higher visual cortex.The natural image generatoris applied to generate reconstructed images from the predicted latent features of natural images and the response signals from the visual cortex.Quantitative and qualitative evaluations were conducted with test images.The results showed that the reconstructed image achieved comparable,accurate reproduction of the presented image in both highlevel semantic category information and low-level pixel information.The framework we propose shows promise for decoding the brain activity.