Exploring the Brain-like Properties of Deep Neural Networks:A Neural Encoding Perspective

Nowadays,deep neural networks(DNNs)have been equipped with powerful representation capabilities.The deep convolutional neural networks(CNNs)that draw inspiration from the visual processing mechanism of the primate early visual cortex have outperformed humans on object categorization and have been found to possess many brain-like properties.Recently,vision transformers(ViTs)have been striking paradigms of DNNs and have achieved remarkable improvements on many vision tasks compared to CNNs.It is natural to ask how the brain-like properties of ViTs are.Beyond the model paradigm,we are also interested in the effects of factors,such as model size,multimodality,and temporality,on the ability of networks to model the human visual pathway,especially when considering that existing research has been limited to CNNs.In this paper,we systematically evaluate the brain-like properties of 30 kinds of computer vision models varying from CNNs and ViTs to their hybrids from the perspective of explaining brain activities of the human visual cortex triggered by dynamic stimuli.Experiments on two neural datasets demonstrate that neither CNN nor transformer is the optimal model paradigm for modelling the human visual pathway.ViTs reveal hierarchical correspondences to the visual pathway as CNNs do.Moreover,we find that multi-modal and temporal networks can better explain the neural activities of large parts of the visual cortex,whereas a larger model size is not a sufficient condition for bridging the gap between human vision and artificial networks.Our study sheds light on the design principles for more brain-like networks.The code is available at https://github.com/QYiZhou/LWNeuralEncoding.

Stretchable,skin-conformable neuromorphic system for tactile sensory recognizing and encoding

Expanding wearable technologies to artificial tactile perception will be of significance for intelligent human-machine interface,as neuromorphic sensing devices are promising candidates due to their low energy consumption and highly effective operating properties.Skin-compatible and conformable features are required for the purpose of realizing wearable artificial tactile perception.Here,we report an intrinsically stretchable,skin-integrated neuromorphic system with triboelectric nanogenerators as tactile sensing and organic electrochemical transistors as information processing.The integrated system provides desired sensing,synaptic,and mechanical characteristics,such as sensitive response(~0.04 kPa^(-1))to low-pressure,short-and long-term synaptic plasticity,great switching endurance(>10000 pulses),symmetric weight update,together with high stretchability of 100%strain.With neural encoding,demonstrations are capable of recognizing,extracting,and encoding features of tactile information.This work provides a feasible approach to wearable,skin-conformable neuromorphic sensing system with great application prospects in intelligent robotics and replacement prosthetics.

通过深度神经网络与皮层网络的多级整合增强自然感知的神经编码模型

Cognitive neuroscience aims to develop computational models that can accurately predict and explain neural responses to sensory inputs in the cortex. Recent studies attempt to leverage the representation power of deep neural networks(DNNs) to predict the brain response and suggest a correspondence between artificial and biological neural networks in their feature representations. However, typical voxel-wise encoding models tend to rely on specific networks designed for computer vision tasks, leading to suboptimal brain-wide correspondence during cognitive tasks. To address this challenge, this work proposes a novel approach that upgrades voxel-wise encoding models through multi-level integration of features from DNNs and information from brain networks. Our approach combines DNN feature-level ensemble learning and brain atlas-level model integration, resulting in significant improvements in predicting whole-brain neural activity during naturalistic video perception. Furthermore, this multi-level integration framework enables a deeper understanding of the brain's neural representation mechanism, accurately predicting the neural response to complex visual concepts. We demonstrate that neural encoding models can be optimized by leveraging a framework that integrates both data-driven approaches and theoretical insights into the functional structure of the cortical networks.