Multimodal Integration Processes in Plan-Based Service Robot Control

Cross-modal integration processes are essential for service robots to reliably perceive relevant parts of the partially known unstructured environment, We demonstrate how multimodal integration on different abstraction levels leads to reasonable behavior that would be difficult to achieve with unimodal approaches. Sensing and acting modalities are composed to multimodal robot skills via a fuzzy multisensor fusion approach. Single modalities constitute basic robot skills that can dynamically be composed to appropriate behavior by symbolic planning. Furthermore, multimodal integration is exploited to answer relevant queries about the partially known environment. All these approaches are successfully implemented and tested on our mobile service robot platform TASER,

Multimodal integrated intervention for children with attentiondeficit/hyperactivity disorder

BACKGROUND Attention-deficit/hyperactivity disorder(ADHD)is one of the most common disorders in child and adolescent psychiatry,with a prevalence of more than 5%.Despite extensive research on ADHD in the last 10 to 20 years,effective treatments are still lacking.Instead,the concept of ADHD seems to have become broader and more heterogeneous.Therefore,the diagnosis and treatment of ADHD remains challenging for clinicians.AIM To investigate the effects of a multimodal integrated intervention for children with ADHD.METHODS Between March 2019 and September 2020,a total of 100 children with ADHD who were diagnosed and treated at our hospital were assessed for eligibility,two of whom revoked their consent.A case-control study was conducted in which the children were equally assigned,using a randomized number table,to either a medication group(methylphenidate hydrochloride extended-release tablets and atomoxetine hydrochloride tablets)or a multimodal integrated intervention group(medication+parent training+behavior modification+sensory integration therapy+sand tray therapy),with 49 patients in each group.The clinical endpoint was the efficacy of the different intervention modalities.RESULTS The two groups of children with ADHD had comparable patient characteristics(P>0.05).Multimodal integrated intervention resulted in a significantly higher treatment efficacy(91.84%)than medication alone(75.51%)(P<0.05).Children who received the multimodal integrated intervention showed lower scores in the Conners Parent Symptom Questionnaire and the Weiss Functional Impairment Rating Scale than those treated with medication alone(P<0.05).The Sensory Integration Scale scores of children in the multimodal integrated intervention group were higher than those of children in the medication group(P<0.05).Children who received the multimodal integrated intervention had higher compliance and family satisfaction and a lower incidence of adverse events than those treated with medication alone(P<0.05).CONCLUSION Multimodal integrated intervention effectively alleviated symptoms associated with ADHD in children.It enhanced their memory and attention with high safety and parental satisfaction,demonstrating good potential for clinical promotion.

Recent advances in multimodal sensing integration and decoupling strategies for tactile perception

Human skin perceives external environmental stimulus by the synergies between the subcutaneous tactile corpuscles.Soft electronics with multiple sensing capabilities by mimicking the function of human skin are of significance in health monitoring and artificial sensation.The last decade has witnessed unprecedented development and convergence between multimodal tactile sensing devices and soft bioelectronics.Despite these advances,traditional flexible electronics achieve multimodal tactile sensing for pressure,strain,temperature,and humidity by integrating monomodal sensing devices together.This strategy results in high energy consumption,limited integration,and complex manufacturing process.Various multimodal sensors and crosstalk-free sensing mechanisms have been proposed to bridge the gap between natural sensory system and artificial perceptual system.In this review,we provide a comprehensive summary of tactile sensing mechanism,integration design principles,signal-decoupling strategies,and current applications for multimodal tactile perception.Finally,we highlight the current challenges and present the future perspectives to promote the development of multimodal tactile perception.

Multimodal integrated strategy for the discovery and identification of quality markers in traditional Chinese medicine

With the modernization and internationalization of traditional Chinese medicine(TCM),the requirement for quality control has increased.The quality marker(Q-marker)is an important standard in this field and has been implemented with remarkable success in recent years.However,the establishment of Qmarkers remains fragmented and the process lacks systematicity,resulting in inconsistent quality control and insufficient correlation with clinical efficacy and safety of TCM.This review introduces four multimodal integrated approaches that contribute to the discovery of more comprehensive and accurate Qmarkers,thus aiding in the establishment of new quality control patterns based on the characteristics and principles of TCM.These include the whole-process quality control strategy,chemical-activity-based screening method,efficacy,safety,and consistent combination strategy,and TCM theory-guided approach.Furthermore,methodologies and representative examples of these strategies are described,and important future directions and questions in this field are also proposed.

iCEMIGE: Integration of CEll-morphometrics, MIcrobiome, and GEne biomarker signatures for risk stratification in breast cancers

BACKGROUND The development of precision medicine is essential for personalized treatment and improved clinical outcome,whereas biomarkers are critical for the success of precision therapies.AIM To investigate whether iCEMIGE(integration of CEll-morphometrics,MIcro-biome,and GEne biomarker signatures)improves risk stratification of breast cancer(BC)patients.METHODS We used our recently developed machine learning technique to identify cellular morphometric biomarkers(CMBs)from the whole histological slide images in The Cancer Genome Atlas(TCGA)breast cancer(TCGA-BRCA)cohort.Multivariate Cox regression was used to assess whether cell-morphometrics prognosis score(CMPS)and our previously reported 12-gene expression prognosis score(GEPS)and 15-microbe abundance prognosis score(MAPS)were independent prognostic factors.iCEMIGE was built upon the sparse representation learning technique.The iCEMIGE scoring model performance was measured by the area under the receiver operating characteristic curve compared to CMPS,GEPS,or MAPS alone.Nomogram models were created to predict overall survival(OS)and progress-free survival(PFS)rates at 5-and 10-year in the TCGA-BRCA cohort.RESULTS We identified 39 CMBs that were used to create a CMPS system in BCs.CMPS,GEPS,and MAPS were found to be significantly independently associated with OS.We then established an iCEMIGE scoring system for risk stratification of BC patients.The iGEMIGE score has a significant prognostic value for OS and PFS independent of clinical factors(age,stage,and estrogen and progesterone receptor status)and PAM50-based molecular subtype.Importantly,the iCEMIGE score significantly increased the power to predict OS and PFS compared to CMPS,GEPS,or MAPS alone.CONCLUSION Our study demonstrates a novel and generic artificial intelligence framework for multimodal data integration toward improving prognosis risk stratification of BC patients,which can be extended to other types of cancer.

TIST: Transcriptome and Histopathological Image Integrative Analysis for Spatial Transcriptomics

Sequencing-based spatial transcriptomics(ST)is an emerging technology to study in situ gene expression patterns at the whole-genome scale.Currently,ST data analysis is still complicated by high technical noises and low resolution.In addition to the transcriptomic data,matched histopathological images are usually generated for the same tissue sample along the ST experiment.The matched high-resolution histopathological images provide complementary cellular phenotypical information,providing an opportunity to mitigate the noises in ST data.We present a novel ST data analysis method called transcriptome and histopathological image integrative analysis for ST(TIST),which enables the identification of spatial clusters(SCs)and the enhancement of spatial gene expression patterns by integrative analysis of matched transcriptomic data and images.TIST devises a histopathological feature extraction method based on Markov random field(MRF)to learn the cellular features from histopathological images,and integrates them with the transcriptomic data and location information as a network,termed TIST-net.Based on TIST-net,SCs are identified by a random walk-based strategy,and gene expression patterns are enhanced by neighborhood smoothing.We benchmark TIST on both simulated datasets and 32 real samples against several state-of-the-art methods.Results show that TIST is robust to technical noises on multiple analysis tasks for sequencing-based ST data and can find interesting microstructures in different biological scenarios.TIST is available at http://lifeome.net/software/tist/and https://ngdc.cncb.ac.cn/biocode/tools/BT007317.