Use of sensory substitution devices as a model system for investigating cross-modal neuroplasticity in humans

Blindness provides an unparalleled opportunity to study plasticity of the nervous system in humans.Seminal work in this area examined the often dramatic modifications to the visual cortex that result when visual input is completely absent from birth or very early in life（Kupers and Ptito,2014）.More recent studies explored what happens to the visual pathways in the context of acquired blindness.This is particularly relevant as the majority of diseases that cause vision loss occur in the elderly.

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Multimodal sentiment analysis aims to understand people’s emotions and opinions from diverse data.Concate-nating or multiplying various modalities is a traditional multi-modal sentiment analysis fusion method.This fusion method does not utilize the correlation information between modalities.To solve this problem,this paper proposes amodel based on amulti-head attention mechanism.First,after preprocessing the original data.Then,the feature representation is converted into a sequence of word vectors and positional encoding is introduced to better understand the semantic and sequential information in the input sequence.Next,the input coding sequence is fed into the transformer model for further processing and learning.At the transformer layer,a cross-modal attention consisting of a pair of multi-head attention modules is employed to reflect the correlation between modalities.Finally,the processed results are input into the feedforward neural network to obtain the emotional output through the classification layer.Through the above processing flow,the model can capture semantic information and contextual relationships and achieve good results in various natural language processing tasks.Our model was tested on the CMU Multimodal Opinion Sentiment and Emotion Intensity(CMU-MOSEI)and Multimodal EmotionLines Dataset(MELD),achieving an accuracy of 82.04% and F1 parameters reached 80.59% on the former dataset.

Cross-Modal Consistency with Aesthetic Similarity for Multimodal False Information Detection

With the explosive growth of false information on social media platforms, the automatic detection of multimodalfalse information has received increasing attention. Recent research has significantly contributed to multimodalinformation exchange and fusion, with many methods attempting to integrate unimodal features to generatemultimodal news representations. However, they still need to fully explore the hierarchical and complex semanticcorrelations between different modal contents, severely limiting their performance detecting multimodal falseinformation. This work proposes a two-stage detection framework for multimodal false information detection,called ASMFD, which is based on image aesthetic similarity to segment and explores the consistency andinconsistency features of images and texts. Specifically, we first use the Contrastive Language-Image Pre-training(CLIP) model to learn the relationship between text and images through label awareness and train an imageaesthetic attribute scorer using an aesthetic attribute dataset. Then, we calculate the aesthetic similarity betweenthe image and related images and use this similarity as a threshold to divide the multimodal correlation matrixinto consistency and inconsistencymatrices. Finally, the fusionmodule is designed to identify essential features fordetectingmultimodal false information. In extensive experiments on four datasets, the performance of the ASMFDis superior to state-of-the-art baseline methods.

Aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders:progress of experimental models based on disease pathogenesis

Neuromyelitis optica spectrum disorders are neuroinflammatory demyelinating disorders that lead to permanent visual loss and motor dysfunction.To date,no effective treatment exists as the exact causative mechanism remains unknown.Therefore,experimental models of neuromyelitis optica spectrum disorders are essential for exploring its pathogenesis and in screening for therapeutic targets.Since most patients with neuromyelitis optica spectrum disorders are seropositive for IgG autoantibodies against aquaporin-4,which is highly expressed on the membrane of astrocyte endfeet,most current experimental models are based on aquaporin-4-IgG that initially targets astrocytes.These experimental models have successfully simulated many pathological features of neuromyelitis optica spectrum disorders,such as aquaporin-4 loss,astrocytopathy,granulocyte and macrophage infiltration,complement activation,demyelination,and neuronal loss;however,they do not fully capture the pathological process of human neuromyelitis optica spectrum disorders.In this review,we summarize the currently known pathogenic mechanisms and the development of associated experimental models in vitro,ex vivo,and in vivo for neuromyelitis optica spectrum disorders,suggest potential pathogenic mechanisms for further investigation,and provide guidance on experimental model choices.In addition,this review summarizes the latest information on pathologies and therapies for neuromyelitis optica spectrum disorders based on experimental models of aquaporin-4-IgG-seropositive neuromyelitis optica spectrum disorders,offering further therapeutic targets and a theoretical basis for clinical trials.

Exploiting fly models to investigate rare human neurological disorders

Rare neurological diseases,while individually are rare,collectively impact millions globally,leading to diverse and often severe neurological symptoms.Often attributed to genetic mutations that disrupt protein function or structure,understanding their genetic basis is crucial for accurate diagnosis and targeted therapies.To investigate the underlying pathogenesis of these conditions,researchers often use non-mammalian model organisms,such as Drosophila(fruit flies),which is valued for their genetic manipulability,cost-efficiency,and preservation of genes and biological functions across evolutionary time.Genetic tools available in Drosophila,including CRISPR-Cas9,offer a means to manipulate gene expression,allowing for a deep exploration of the genetic underpinnings of rare neurological diseases.Drosophila boasts a versatile genetic toolkit,rapid generation turnover,and ease of large-scale experimentation,making it an invaluable resource for identifying potential drug candidates.Researchers can expose flies carrying disease-associated mutations to various compounds,rapidly pinpointing promising therapeutic agents for further investigation in mammalian models and,ultimately,clinical trials.In this comprehensive review,we explore rare neurological diseases where fly research has significantly contributed to our understanding of their genetic basis,pathophysiology,and potential therapeutic implications.We discuss rare diseases associated with both neuron-expressed and glial-expressed genes.Specific cases include mutations in CDK19 resulting in epilepsy and developmental delay,mutations in TIAM1 leading to a neurodevelopmental disorder with seizures and language delay,and mutations in IRF2BPL causing seizures,a neurodevelopmental disorder with regression,loss of speech,and abnormal movements.And we explore mutations in EMC1 related to cerebellar atrophy,visual impairment,psychomotor retardation,and gain-of-function mutations in ACOX1 causing Mitchell syndrome.Loss-of-function mutations in ACOX1 result in ACOX1 deficiency,characterized by very-long-chain fatty acid accumulation and glial degeneration.Notably,this review highlights how modeling these diseases in Drosophila has provided valuable insights into their pathophysiology,offering a platform for the rapid identification of potential therapeutic interventions.Rare neurological diseases involve a wide range of expression systems,and sometimes common phenotypes can be found among different genes that cause abnormalities in neurons or glia.Furthermore,mutations within the same gene may result in varying functional outcomes,such as complete loss of function,partial loss of function,or gain-of-function mutations.The phenotypes observed in patients can differ significantly,underscoring the complexity of these conditions.In conclusion,Drosophila represents an indispensable and cost-effective tool for investigating rare neurological diseases.By facilitating the modeling of these conditions,Drosophila contributes to a deeper understanding of their genetic basis,pathophysiology,and potential therapies.This approach accelerates the discovery of promising drug candidates,ultimately benefiting patients affected by these complex and understudied diseases.

A promising approach for quantifying focal stroke modeling and assessing stroke progression:optical resolution photoacoustic microscopy photothrombosis

To investigate the mechanisms underlying the onset and progression of ischemic stroke,some methods have been proposed that can simultaneously monitor and create embolisms in the animal cerebral cortex.However,these methods often require complex systems and the effect of age on cerebral embolism has not been adequately studied,although ischemic stroke is strongly age-related.In this study,we propose an optical-resolution photoacoustic microscopy-based visualized photothrombosis methodology to create and monitor ischemic stroke in mice simultaneously using a 532 nm pulsed laser.We observed the molding process in mice of different ages and presented age-dependent vascular embolism differentiation.Moreover,we integrated optical coherence tomography angiography to investigate age-associated trends in cerebrovascular variability following a stroke.Our imaging data and quantitative analyses underscore the differential cerebrovascular responses to stroke in mice of different ages,thereby highlighting the technique's potential for evaluating cerebrovascular health and unraveling age-related mechanisms involved in ischemic strokes.

Reduced mesencephalic astrocyte-derived neurotrophic factor expression by mutant androgen receptor contributes to neurodegeneration in a model of spinal and bulbar muscular atrophy pathology

Spinal and bulbar muscular atrophy is a neurodegenerative disease caused by extended CAG trinucleotide repeats in the androgen receptor gene,which encodes a ligand-dependent transcription facto r.The mutant androgen receptor protein,characterized by polyglutamine expansion,is prone to misfolding and forms aggregates in both the nucleus and cytoplasm in the brain in spinal and bulbar muscular atrophy patients.These aggregates alter protein-protein interactions and compromise transcriptional activity.In this study,we reported that in both cultured N2a cells and mouse brain,mutant androgen receptor with polyglutamine expansion causes reduced expression of mesencephalic astrocyte-de rived neurotrophic factor.Overexpressio n of mesencephalic astrocyte-derived neurotrophic factor amelio rated the neurotoxicity of mutant androgen receptor through the inhibition of mutant androgen receptor aggregation.Conversely.knocking down endogenous mesencephalic astrocyte-derived neurotrophic factor in the mouse brain exacerbated neuronal damage and mutant androgen receptor aggregation.Our findings suggest that inhibition of mesencephalic astrocyte-derived neurotrophic factor expression by mutant androgen receptor is a potential mechanism underlying neurodegeneration in spinal and bulbar muscular atrophy.

采用STAMP-24Model的多组织事故分析

安全生产事故往往由多组织交互、多因素耦合造成,事故原因涉及多个组织。为预防和遏制多组织生产安全事故的发生,基于系统理论事故建模与过程模型(Systems-Theory Accident Modeling and Process,STAMP)、24Model,构建一种用于多组织事故分析的方法,并以青岛石油爆炸事故为例进行事故原因分析。结果显示:STAMP-24Model可以分组织,分层次且有效、全面、详细地分析涉及多个组织的事故原因,探究多组织之间的交互关系;对事故进行动态演化分析,可得到各组织不安全动作耦合关系与形成的事故失效链及管控失效路径,进而为预防多组织事故提供思路和参考。

基于改进24Model-ISM-SNA建筑工人不安全行为关联路径研究

建筑施工现场环境复杂,为有效控制不安全行为发生,基于行为安全“2-4”模型对360份具有代表性的建筑安全事故调查报告进行分析,提取出22个不安全行为的主要影响因素。利用灰色关联分析方法(GRA)改进的集成ISM-SNA模型,将不安全行为风险因素划分为表层、过渡层与深层,然后对风险因素进行可视化分析、中心度分析及凝聚子群分析,揭示了各致因因素间的关联关系和传导路径。结果表明,建筑工人不安全行为影响因素可划分成7级3阶的多级递阶结构,安全意识、现场监管、外部环境是建筑工人不安全行为的关键影响因素,同时现场监管和隐患排查到位能有效降低不安全行为的发生。

基于24Model的地铁内涝事故原因分析与评估

为降低地铁内涝事故灾害风险,基于事故致因“2-4”模型,分析了地铁内涝事故致灾因子,采用层次分析法构建了地铁内涝事故原因分析指标体系,确定了各风险因子的权重,并利用模糊综合评价法对地铁内涝事故进行了定量评估,识别出关键的影响因素。结果表明:地铁内涝事故一级指标中不安全动作与物态因素最重要,其中影响最大的指标包括擅自更改建筑设计、未按照要求检查水位情况、未及时排水、出入口不符合防汛标准等因素;习惯性不安全行为的权重位居第二,表明该指标因素较为重要,同时安全管理体系得分位居第二,表明该指标因素较易发生。对关键指标采取防范措施,可有效降低风险,从而减少地铁内涝事故的发生。

基于24Model-D-ISM的地铁站火灾疏散影响因素研究

为预防地铁站火灾事故,深入了解地铁站火灾人员疏散影响因素间的内在联系与层次结构,基于第6版“2-4”模型(24Model)分析63起地铁站火灾疏散事故,充分考虑各个因素之间的交互作用,提取19个影响地铁站人员疏散的关键因素,建立地铁站火灾人员疏散影响因素指标体系;采用算子客观赋权法(C-OWA)改进决策试验与评价实验法(DEMATEL),确定地铁站火灾人员疏散的重要影响因素;在此基础上,采用解释结构模型(ISM)分析各个因素间的层次结构及相互作用路径,构建地铁站火灾人员疏散影响因素的多级递阶结构模型。研究结果表明:疏散引导、恐慌从众行为、人员拥挤为地铁站火灾人员疏散的关键影响因素;地铁站火灾人员疏散受表层因素、中间层因素、深层因素共同作用的影响,其中,疏散教育与培训、设施维护与检查、疏散预案等因素是根源影响因素,重视根源影响因素的改善有利于从本质上预防和控制事故的发生。

A Multi-Level Circulant Cross-Modal Transformer for Multimodal Speech Emotion Recognition

Speech emotion recognition,as an important component of humancomputer interaction technology,has received increasing attention.Recent studies have treated emotion recognition of speech signals as a multimodal task,due to its inclusion of the semantic features of two different modalities,i.e.,audio and text.However,existing methods often fail in effectively represent features and capture correlations.This paper presents a multi-level circulant cross-modal Transformer(MLCCT)formultimodal speech emotion recognition.The proposed model can be divided into three steps,feature extraction,interaction and fusion.Self-supervised embedding models are introduced for feature extraction,which give a more powerful representation of the original data than those using spectrograms or audio features such as Mel-frequency cepstral coefficients(MFCCs)and low-level descriptors(LLDs).In particular,MLCCT contains two types of feature interaction processes,where a bidirectional Long Short-term Memory(Bi-LSTM)with circulant interaction mechanism is proposed for low-level features,while a two-stream residual cross-modal Transformer block is appliedwhen high-level features are involved.Finally,we choose self-attention blocks for fusion and a fully connected layer to make predictions.To evaluate the performance of our proposed model,comprehensive experiments are conducted on three widely used benchmark datasets including IEMOCAP,MELD and CMU-MOSEI.The competitive results verify the effectiveness of our approach.

Mechanism of Cross-modal Information Influencing Taste

Studies on the integration of cross-modal information with taste perception has been mostly limited to uni-modal level.The cross-modal sensory interaction and the neural network of information processing and its control were not fully explored and the mechanisms remain poorly understood.This mini review investigated the impact of uni-modal and multi-modal information on the taste perception,from the perspective of cognitive status,such as emotion,expectation and attention,and discussed the hypothesis that the cognitive status is the key step for visual sense to exert influence on taste.This work may help researchers better understand the mechanism of cross-modal information processing and further develop neutrally-based artificial intelligent(AI)system.

24Model与LCM原因因素定义对比研究

为探究损失致因模型(LCM)原因因素定义与事故致因“2-4”模型(24Model)存在的异同和优缺点,梳理2个模型各层面原因和结果的定义,对比定义内容及其对事故原因分析等安全实务的指导作用,并以一起瓦斯爆炸事故为例加以实证分析,获得二者分析结果之间的差异。研究结果表明:LCM是首个将管理因素纳入事故致因分析的一维事件序列模型,可明确各层面原因因素的定义和因素间的逻辑关系,但部分定义存在交叉重复的问题,并没有揭示安全工作指导思想等深层次事故致因因素;24Model作为系统性事故致因模型,对各类因素的定义均以组织为主体,描述事件、事故、安全的概念内涵,划分个体安全动作、安全能力和组织安全管理体系的类别并给出含义解析,探究组织安全文化层面的问题并以32个元素体现;2个模型的事故原因分析方法均建立在对各层级原因因素定义的基础上,并适用于模型理论体系本身。

CSMCCVA:Framework of cross-modal semantic mapping based on cognitive computing of visual and auditory sensations

Cross-modal semantic mapping and cross-media retrieval are key problems of the multimedia search engine.This study analyzes the hierarchy,the functionality,and the structure in the visual and auditory sensations of cognitive system,and establishes a brain-like cross-modal semantic mapping framework based on cognitive computing of visual and auditory sensations.The mechanism of visual-auditory multisensory integration,selective attention in thalamo-cortical,emotional control in limbic system and the memory-enhancing in hippocampal were considered in the framework.Then,the algorithms of cross-modal semantic mapping were given.Experimental results show that the framework can be effectively applied to the cross-modal semantic mapping,and also provides an important significance for brain-like computing of non-von Neumann structure.

Cross-Modal Hashing Retrieval Based on Deep Residual Network

In the era of big data rich inWe Media,the single mode retrieval system has been unable to meet people’s demand for information retrieval.This paper proposes a new solution to the problem of feature extraction and unified mapping of different modes:A Cross-Modal Hashing retrieval algorithm based on Deep Residual Network(CMHR-DRN).The model construction is divided into two stages:The first stage is the feature extraction of different modal data,including the use of Deep Residual Network(DRN)to extract the image features,using the method of combining TF-IDF with the full connection network to extract the text features,and the obtained image and text features used as the input of the second stage.In the second stage,the image and text features are mapped into Hash functions by supervised learning,and the image and text features are mapped to the common binary Hamming space.In the process of mapping,the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval.In training the model,adaptive moment estimation(Adam)is used to calculate the adaptive learning rate of each parameter,and the stochastic gradient descent(SGD)is calculated to obtain the minimum loss function.The whole training process is completed on Caffe deep learning framework.Experiments show that the proposed algorithm CMHR-DRN based on Deep Residual Network has better retrieval performance and stronger advantages than other Cross-Modal algorithms CMFH,CMDN and CMSSH.

Anisotropic time-dependent behaviors of shale under direct shearing and associated empirical creep models

Understanding the anisotropic creep behaviors of shale under direct shearing is a challenging issue.In this context,we conducted shear-creep and steady-creep tests on shale with five bedding orientations (i.e.0°,30°,45°,60°,and 90°),under multiple levels of direct shearing for the first time.The results show that the anisotropic creep of shale exhibits a significant stress-dependent behavior.Under a low shear stress,the creep compliance of shale increases linearly with the logarithm of time at all bedding orientations,and the increase depends on the bedding orientation and creep time.Under high shear stress conditions,the creep compliance of shale is minimal when the bedding orientation is 0°,and the steady-creep rate of shale increases significantly with increasing bedding orientations of 30°,45°,60°,and 90°.The stress-strain values corresponding to the inception of the accelerated creep stage show an increasing and then decreasing trend with the bedding orientation.A semilogarithmic model that could reflect the stress dependence of the steady-creep rate while considering the hardening and damage process is proposed.The model minimizes the deviation of the calculated steady-state creep rate from the observed value and reveals the behavior of the bedding orientation's influence on the steady-creep rate.The applicability of the five classical empirical creep models is quantitatively evaluated.It shows that the logarithmic model can well explain the experimental creep strain and creep rate,and it can accurately predict long-term shear creep deformation.Based on an improved logarithmic model,the variations in creep parameters with shear stress and bedding orientations are discussed.With abovementioned findings,a mathematical method for constructing an anisotropic shear creep model of shale is proposed,which can characterize the nonlinear dependence of the anisotropic shear creep behavior of shale on the bedding orientation.

TECMH:Transformer-Based Cross-Modal Hashing For Fine-Grained Image-Text Retrieval

In recent years,cross-modal hash retrieval has become a popular research field because of its advantages of high efficiency and low storage.Cross-modal retrieval technology can be applied to search engines,crossmodalmedical processing,etc.The existing main method is to use amulti-label matching paradigm to finish the retrieval tasks.However,such methods do not use fine-grained information in the multi-modal data,which may lead to suboptimal results.To avoid cross-modal matching turning into label matching,this paper proposes an end-to-end fine-grained cross-modal hash retrieval method,which can focus more on the fine-grained semantic information of multi-modal data.First,the method refines the image features and no longer uses multiple labels to represent text features but uses BERT for processing.Second,this method uses the inference capabilities of the transformer encoder to generate global fine-grained features.Finally,in order to better judge the effect of the fine-grained model,this paper uses the datasets in the image text matching field instead of the traditional label-matching datasets.This article experiment on Microsoft COCO(MS-COCO)and Flickr30K datasets and compare it with the previous classicalmethods.The experimental results show that this method can obtain more advanced results in the cross-modal hash retrieval field.

Projecting Wintertime Newly Formed Arctic Sea Ice through Weighting CMIP6 Model Performance and Independence

Precipitous Arctic sea-ice decline and the corresponding increase in Arctic open-water areas in summer months give more space for sea-ice growth in the subsequent cold seasons. Compared to the decline of the entire Arctic multiyear sea ice,changes in newly formed sea ice indicate more thermodynamic and dynamic information on Arctic atmosphere–ocean–ice interaction and northern mid–high latitude atmospheric teleconnections. Here, we use a large multimodel ensemble from phase 6 of the Coupled Model Intercomparison Project(CMIP6) to investigate future changes in wintertime newly formed Arctic sea ice. The commonly used model-democracy approach that gives equal weight to each model essentially assumes that all models are independent and equally plausible, which contradicts with the fact that there are large interdependencies in the ensemble and discrepancies in models' performances in reproducing observations. Therefore, instead of using the arithmetic mean of well-performing models or all available models for projections like in previous studies, we employ a newly developed model weighting scheme that weights all models in the ensemble with consideration of their performance and independence to provide more reliable projections. Model democracy leads to evident bias and large intermodel spread in CMIP6 projections of newly formed Arctic sea ice. However, we show that both the bias and the intermodel spread can be effectively reduced by the weighting scheme. Projections from the weighted models indicate that wintertime newly formed Arctic sea ice is likely to increase dramatically until the middle of this century regardless of the emissions scenario.Thereafter, it may decrease(or remain stable) if the Arctic warming crosses a threshold(or is extensively constrained).

ViT2CMH:Vision Transformer Cross-Modal Hashing for Fine-Grained Vision-Text Retrieval

In recent years,the development of deep learning has further improved hash retrieval technology.Most of the existing hashing methods currently use Convolutional Neural Networks(CNNs)and Recurrent Neural Networks(RNNs)to process image and text information,respectively.This makes images or texts subject to local constraints,and inherent label matching cannot capture finegrained information,often leading to suboptimal results.Driven by the development of the transformer model,we propose a framework called ViT2CMH mainly based on the Vision Transformer to handle deep Cross-modal Hashing tasks rather than CNNs or RNNs.Specifically,we use a BERT network to extract text features and use the vision transformer as the image network of the model.Finally,the features are transformed into hash codes for efficient and fast retrieval.We conduct extensive experiments on Microsoft COCO(MS-COCO)and Flickr30K,comparing with baselines of some hashing methods and image-text matching methods,showing that our method has better performance.