Nomogram based on multimodal magnetic resonance combined with B7-H3mRNA for preoperative lymph node prediction in esophagus cancer

Accurate preoperative prediction of lymph node metastasis(LNM)in esophageal cancer(EC)patients is of crucial clinical significance for treatment planning and prognosis.AIM To develop a clinical radiomics nomogram that can predict the preoperative lymph node(LN)status in EC patients.METHODS A total of 32 EC patients confirmed by clinical pathology(who underwent surgical treatment)were included.Real-time fluorescent quantitative reverse transcription-polymerase chain reaction was used to detect the expression of B7-H3 mRNA in EC tissue obtained during preoperative gastroscopy,and its correlation with LNM was analyzed.Radiomics features were extracted from multi-modal magnetic resonance imaging of EC using Pyradiomics in Python.Feature extraction,data dimensionality reduction,and feature selection were performed using XGBoost model and leave-one-out cross-validation.Multivariable logistic regression analysis was used to establish the prediction model,which included radiomics features,LN status from computed tomography(CT)reports,and B7-H3 mRNA expression,represented by a radiomics nomogram.Receiver operating characteristic area under the curve(AUC)and decision curve analysis(DCA)were used to evaluate the predictive performance and clinical application value of the model.RESULTS The relative expression of B7-H3 mRNA in EC patients with LNM was higher than in those without metastasis,and the difference was statistically significant(P<0.05).The AUC value in the receiver operating characteristic(ROC)curve was 0.718(95%CI:0.528-0.907),with a sensitivity of 0.733 and specificity of 0.706,indicating good diagnostic performance.The individualized clinical prediction nomogram included radiomics features,LN status from CT reports,and B7-H3 mRNA expression.The ROC curve demonstrated good diagnostic value,with an AUC value of 0.765(95%CI:0.598-0.931),sensitivity of 0.800,and specificity of 0.706.DCA indicated the practical value of the radiomics nomogram in clinical practice.CONCLUSION This study developed a radiomics nomogram that includes radiomics features,LN status from CT reports,and B7-H3 mRNA expression,enabling convenient preoperative individualized prediction of LNM in EC patients.

Multimodal Imaging with 3D-Holograms for Preoperative Planning in Pediatric Cardiac Surgery:A Unique Case Report

Multimodal imaging,including augmented or mixed reality,transforms the physicians’interaction with clinical imaging,allowing more accurate data interpretation,better spatial resolution,and depth perception of the patient’s anatomy.We successfully overlay 3D holographic visualization to magnetic resonance imaging images for preoperative decision making of a complex case of cardiac tumour in a 7-year-old girl.

3D Vehicle Detection Algorithm Based onMultimodal Decision-Level Fusion

3D vehicle detection based on LiDAR-camera fusion is becoming an emerging research topic in autonomous driving.The algorithm based on the Camera-LiDAR object candidate fusion method(CLOCs)is currently considered to be a more effective decision-level fusion algorithm,but it does not fully utilize the extracted features of 3D and 2D.Therefore,we proposed a 3D vehicle detection algorithm based onmultimodal decision-level fusion.First,project the anchor point of the 3D detection bounding box into the 2D image,calculate the distance between 2D and 3D anchor points,and use this distance as a new fusion feature to enhance the feature redundancy of the network.Subsequently,add an attention module:squeeze-and-excitation networks,weight each feature channel to enhance the important features of the network,and suppress useless features.The experimental results show that the mean average precision of the algorithm in the KITTI dataset is 82.96%,which outperforms previous state-ofthe-art multimodal fusion-based methods,and the average accuracy in the Easy,Moderate and Hard evaluation indicators reaches 88.96%,82.60%,and 77.31%,respectively,which are higher compared to the original CLOCs model by 1.02%,2.29%,and 0.41%,respectively.Compared with the original CLOCs algorithm,our algorithm has higher accuracy and better performance in 3D vehicle detection.

MFF-Net: Multimodal Feature Fusion Network for 3D Object Detection

In complex traffic environment scenarios,it is very important for autonomous vehicles to accurately perceive the dynamic information of other vehicles around the vehicle in advance.The accuracy of 3D object detection will be affected by problems such as illumination changes,object occlusion,and object detection distance.To this purpose,we face these challenges by proposing a multimodal feature fusion network for 3D object detection(MFF-Net).In this research,this paper first uses the spatial transformation projection algorithm to map the image features into the feature space,so that the image features are in the same spatial dimension when fused with the point cloud features.Then,feature channel weighting is performed using an adaptive expression augmentation fusion network to enhance important network features,suppress useless features,and increase the directionality of the network to features.Finally,this paper increases the probability of false detection and missed detection in the non-maximum suppression algo-rithm by increasing the one-dimensional threshold.So far,this paper has constructed a complete 3D target detection network based on multimodal feature fusion.The experimental results show that the proposed achieves an average accuracy of 82.60%on the Karlsruhe Institute of Technology and Toyota Technological Institute(KITTI)dataset,outperforming previous state-of-the-art multimodal fusion networks.In Easy,Moderate,and hard evaluation indicators,the accuracy rate of this paper reaches 90.96%,81.46%,and 75.39%.This shows that the MFF-Net network has good performance in 3D object detection.

Fast Encoding-Decoding of 3D Hyperspectral Images Using a Non-Supervised Multimodal Compression Scheme

We introduce in this paper an extension of the Multimodal Compression technique (MC) for the purpose of coding hyperspectral image sequences. The main idea requires few steps, namely: (1) reducing the size of the sequence by inserting smooth images containing less information into the remaining images of the same sequence, (2) then coding the new compacted sequence using 3D-SPIHT algorithm. In this new scheme, called MC-3D-SPIHT, the insertion is achieved only in the contour of each image, according to a non-supervised way, so that one can preserve the Region of Interest (ROI) quality. For this purpose, a mixing function is employed. After the decoding process, inserted images are extracted by a separation function and the original sequence is reconstructed. By considering data from AVIRIS database, we will show how one decrease significantly the computing time for both coding and decoding.

基于高光谱图像和3D-CNN的苹果多品质参数无损检测

[目的]为解决水果品质无损检测中成本、效率、精度问题,提出了一种基于高光谱图像和三维卷积神经网络(3D-CNN)的苹果高光谱多品质参数同时检测方法。[方法]使用高光谱成像系统获取400~1000 nm波段的苹果样本的高光谱反射图像并使用S-G平滑法对原始图像进行去噪处理,在此基础上,对采集到的高光谱图像通过多感兴趣位置的选取以及间隔波段抽取重组的方法进行样本扩充,再利用三维卷积神经网络建立样本扩充后的苹果高光谱图像与苹果糖度、硬度、含水量的多任务学习模型,通过该模型实现对苹果的糖度、硬度、含水量等品质参数的无损检测。[结果]采集245个苹果的高光谱图像及其对应的品质参数信息,通过样本扩充的方法将原始数据集扩充至9800个样本后进行建模和验证。结果表明:本算法建立的苹果糖度、硬度、水分的分类模型,在糖度类间隔为1°Brix、硬度类间隔为0.5 kg·cm-2、含水量类间隔为10%的情况下,糖度、硬度、水分的预测准确率分别为93.97%、92.29%和93.36%,回归模型糖度、硬度和水分的相关系数最高分别达到0.827、0.775和0.862,比最优的传统算法分别提高15.0%、17.0%和17.2%。[结论]本算法能够较准确实现苹果高光谱多品质参数同时检测,且相对传统方法预测精度有较大提升。

结合MRI多模态信息和3D-CNNs特征提取的脑肿瘤分割研究

目的探究结合MRI多模态信息和3D-CNNs特征提取对于脑肿瘤分割的价值。方法分析相比于未加入多模态3D-CNNs特征的方法,并对比2D-CNNs特征方法和3D-CNNs特征方法分割的结果,主要参考dice系数,假阳性率和sensitibity。结果在加入多模态3D-CNNs特征之后,患者的dice系数均有不同程度的提高,sensitibity系数也有改变,假阳性率显著得到改善;加上多模态3D-CNNs特征提取后,dice系数变为(88.26±4.65)%,显著优于多模态2D-CNNs特征提取的(83.67±4.22)%。而多模态2D-CNNs特征提取的运用甚至比单独使用灰度邻域结合haar小波低频系数的分割结果。结论基于多模态3D-CNNs特征提取的MRI脑肿瘤分割准确度高,适应不同患者不同模态之间的多变性和差异性,值得参考。

快速3D-CNN结合深度可分离卷积对高光谱图像分类

针对卷积神经网络在高光谱图像特征提取和分类的过程中,存在空谱特征提取不充分以及网络层数太多引起的参数量大、计算复杂的问题,提出快速三维卷积神经网络(3D-CNN)结合深度可分离卷积(DSC)的轻量型卷积模型。该方法首先利用增量主成分分析(IPCA)对输入的数据进行降维预处理;其次将输入模型的像素分割成小的重叠的三维小卷积块,在分割的小块上基于中心像素形成地面标签,利用三维核函数进行卷积处理,形成连续的三维特征图,保留空谱特征。用3D-CNN同时提取空谱特征,然后在三维卷积中加入深度可分离卷积对空间特征再次提取,丰富空谱特征的同时减少参数量,从而减少计算时间,分类精度也有所提高。所提模型在Indian Pines、Salinas Scene和University of Pavia公开数据集上验证,并且同其他经典的分类方法进行比较。实验结果表明,该方法不仅能大幅度节省可学习的参数,降低模型复杂度,而且表现出较好的分类性能,其中总体精度(OA)、平均分类精度(AA)和Kappa系数均可达99%以上。

联合LiDAR、高光谱数据及3D-CNN方法的树种分类

【目的】探究三维卷积神经网络(3D-CNN)在高光谱数据支持的树种分类中的有效网络构建方式,以提高树种分类精度。【方法】以美国加利福尼亚州内华达山脉南部为研究区,LiDAR数据获取的森林冠层高(CHM)进行单木分割并以此为补充建立样本,改进一种结构更简单、分类精度更高且无需对高光谱数据进行预处理的3D-CNN网络结构用于森林树种识别。【结果】相较于常规机器学习分类方法【支持向量机(SVM),随机森林(RF)】、传统二维卷积神经网络模型(2D-CNN)及最新多光谱分辨率三维卷积神经网络(MSR 3D-CNN)模型,本研究提出的3D-CNN模型对树种总体分类精度为99.79%,平均交并比(MIoU)为99.53%。与SVM和RF分类结果相比,本研究构建的3D-CNN模型总体分类精度提高5%左右,且具有对树种边界提取更加准确、椒盐现象更少发生的特点;与2D-CNN相比,总体分类精度提高10%左右,MIoU提高7%左右;与MSR 3D-CNN相比,总体精度相差不大,但在训练和测试过程中,本模型耗时远远小于MSR 3D-CNN模型。【结论】本研究改进的3D-CNN模型结构能够高效对原始高光谱影像进行树种分类并制图,可有效提高树种分类的精度。

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Recognition of dynamic hand gestures in real-time is a difficult task because the system can never know when or from where the gesture starts and ends in a video stream.Many researchers have been working on visionbased gesture recognition due to its various applications.This paper proposes a deep learning architecture based on the combination of a 3D Convolutional Neural Network(3D-CNN)and a Long Short-Term Memory(LSTM)network.The proposed architecture extracts spatial-temporal information from video sequences input while avoiding extensive computation.The 3D-CNN is used for the extraction of spectral and spatial features which are then given to the LSTM network through which classification is carried out.The proposed model is a light-weight architecture with only 3.7 million training parameters.The model has been evaluated on 15 classes from the 20BN-jester dataset available publicly.The model was trained on 2000 video-clips per class which were separated into 80%training and 20%validation sets.An accuracy of 99%and 97%was achieved on training and testing data,respectively.We further show that the combination of 3D-CNN with LSTM gives superior results as compared to MobileNetv2+LSTM.

Attention Based Multi-Patched 3D-CNNs with Hybrid Fusion Architecture for Reducing False Positives during Lung Nodule Detection

In lung nodules there is a huge variation in structural properties like Shape, Surface Texture. Even the spatial properties vary, where they can be found attached to lung walls, blood vessels in complex non-homogenous lung structures. Moreover, the nodules are of small size at their early stage of development. This poses a serious challenge to develop a Computer aided diagnosis (CAD) system with better false positive reduction. Hence, to reduce the false positives per scan and to deal with the challenges mentioned, this paper proposes a set of three diverse 3D Attention based CNN architectures (3D ACNN) whose predictions on given low dose Volumetric Computed Tomography (CT) scans are fused to achieve more effective and reliable results. Attention mechanism is employed to selectively concentrate/weigh more on nodule specific features and less weight age over other irrelevant features. By using this attention based mechanism in CNN unlike traditional methods there was a significant gain in the classification performance. Contextual dependencies are also taken into account by giving three patches of different sizes surrounding the nodule as input to the ACNN architectures. The system is trained and validated using a publicly available LUNA16 dataset in a 10 fold cross validation approach where a competition performance metric (CPM) score of 0.931 is achieved. The experimental results demonstrate that either a single patch or a single architecture in a one-to-one fashion that is adopted in earlier methods cannot achieve a better performance and signifies the necessity of fusing different multi patched architectures. Though the proposed system is mainly designed for pulmonary nodule detection it can be easily extended to classification tasks of any other 3D medical diagnostic computed tomography images where there is a huge variation and uncertainty in classification.

3D-CNN在肺癌图像识别中的应用研究

肺癌是长期威胁人类健康的恶性疾病之一,针对传统方法在肺癌CT图像分类中的预处理过程复杂、工作量大的问题,本文提出了基于三维卷积神经网络(3D-CNN)模型的肺部CT图像分类方法。该模型以卷积神经网络模型为基础,并在训练的过程中使用特定顺序输入策略,还在公开的Kaggle Data Science Bowl 2017数据集上进行了实验。实验表明,该方法对图像的分类准确率达到76%,比采用随机顺序的输入策略时有所提升,能够为肺部病理图像的分类研究提供有价值的参考。

基于大类招生的《信息技术基础》课程3M教学模式研究

该文提出了针对数物信类学生《信息技术基础》课程的＂3M教学模式＂,给出了3M教学模式的具体含义和信息技术基础课程体系结构。文章阐释了不同内容模块使用的不同教学模式。通过实践得出了最优教学模式为混合式教学模式。使用3M教学模式,能充分发挥多媒体教学手段的优势,化抽象为形象。利用多方式教学手段,不同模块使用不同教学方法,结合慕课与微课,解除时间、空间的限制,将线上学习与线下学习结合起来,提高信息技术基础课程的教学效率。

“3M教学模式”在《信息技术基础》课程教学中的应用与实践

该文将"3M教学模式"应用于《信息技术基础》课程教学中,并结合具体教学情况,给出了3M在教学实践中的应用分析。文章结合课程的内容构架,给出了课程模块的详细内容,还给出了不同模块学时分配与分析。文章结合信息技术基础课程在实践中的应用情况,探讨了该课程的考核方式,给出了考核结果分析。得出"3M教学模式"在教学中的初步应用切实可行。所以,将多模式、多媒体、慕课和微课等方式融合起来,将线上学习与线下学习结合起来,全方位、多维度、广视角的开展信息技术基础的教与学,将有利于信息技术基础课程教学效果的提高。

基于3D-LCRN视频异常行为识别方法

自动准确识别监控视频中的异常行为在安防领域具有广泛的应用前景.本文提出一种基于3D-LCRN(3D Long-short-term Convolutional Recurrent Network)视觉时序模型的视频异常行为识别方法.首先,基于视频图像帧间的结构相似性,结合光照感应与光照补偿机制进行背景建模,获取对光照突变与背景运动不敏感的矫正光流场与矫正运动历史图.同时,针对异常与正常行为视频数据失衡问题,计算三通道矫正光流运动历史图COFMHI(corrected optical flow motion history image),随机提取视觉词块进行聚类,对样本数量与维度进行双向扩充,充分获取样本的微分和积分运动信息.在此基础上,采用3D-CNN深度学习网络模型对COFMHI进行学习,获取局部短时序时空-域特征,结合可学习贡献因子加权的LSTM网络以压制无关、冗余、具有混淆性的视频片段,进一步提取由短时序-长时序,由局部-全局的多层次时-空域特征用于异常行为识别.通过与同类方法的客观定量对比,实验结果表明,本文方法在光照突变与背景运动等复杂场景下具有优异的异常行为识别性能,进一步表明该方法有效、可行.

Enhancing 3D Reconstruction Accuracy of FIB Tomography Data Using Multi‑voltage Images and Multimodal Machine Learning

FIB-SEM tomography is a powerful technique that integrates a focused ion beam(FIB)and a scanning electron microscope(SEM)to capture high-resolution imaging data of nanostructures.This approach involves collecting in-plane SEM imagesand using FIB to remove material layers for imaging subsequent planes,thereby producing image stacks.However,theseimage stacks in FIB-SEM tomography are subject to the shine-through effect,which makes structures visible from theposterior regions of the current plane.This artifact introduces an ambiguity between image intensity and structures in thecurrent plane,making conventional segmentation methods such as thresholding or the k-means algorithm insufficient.Inthis study,we propose a multimodal machine learning approach that combines intensity information obtained at differentelectron beam accelerating voltages to improve the three-dimensional(3D)reconstruction of nanostructures.By treatingthe increased shine-through effect at higher accelerating voltages as a form of additional information,the proposed methodsignificantly improves segmentation accuracy and leads to more precise 3D reconstructions for real FIB tomography data.

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

Today,fatalities,physical injuries,and significant economic losses occur due to car accidents.Among the leading causes of car accidents is drowsiness behind the wheel,which can affect any driver.Drowsiness and sleepiness often have associated indicators that researchers can use to identify and promptly warn drowsy drivers to avoid potential accidents.This paper proposes a spatiotemporal model for monitoring drowsiness visual indicators from videos.This model depends on integrating a 3D convolutional neural network(3D-CNN)and long short-term memory(LSTM).The 3DCNN-LSTM can analyze long sequences by applying the 3D-CNN to extract spatiotemporal features within adjacent frames.The learned features are then used as the input of the LSTM component for modeling high-level temporal features.In addition,we investigate how the training of the proposed model can be affected by changing the position of the batch normalization(BN)layers in the 3D-CNN units.The BN layer is examined in two different placement settings:before the non-linear activation function and after the non-linear activation function.The study was conducted on two publicly available drowsy drivers datasets named 3MDAD and YawDD.3MDAD is mainly composed of two synchronized datasets recorded from the frontal and side views of the drivers.We show that the position of the BN layers increases the convergence speed and reduces overfitting on one dataset but not the other.As a result,the model achieves a test detection accuracy of 96%,93%,and 90%on YawDD,Side-3MDAD,and Front-3MDAD,respectively.

Nature-Inspired Level Set Segmentation Model for 3D-MRI Brain Tumor Detection

Medical image segmentation has consistently been a significant topic of research and a prominent goal,particularly in computer vision.Brain tumor research plays a major role in medical imaging applications by providing a tremendous amount of anatomical and functional knowledge that enhances and allows easy diagnosis and disease therapy preparation.To prevent or minimize manual segmentation error,automated tumor segmentation,and detection became the most demanding process for radiologists and physicians as the tumor often has complex structures.Many methods for detection and segmentation presently exist,but all lack high accuracy.This paper’s key contribution focuses on evaluating machine learning techniques that are supposed to reduce the effect of frequently found issues in brain tumor research.Furthermore,attention concentrated on the challenges related to level set segmentation.The study proposed in this paper uses the Population-based Artificial Bee Colony Clustering(P-ABCC)methodology to reliably collect initial contour points,which helps minimize the number of iterations and segmentation errors of the level-set process.The proposed model measures cluster centroids(ABC populations)and uses a level-set approach to resolve contour differences as brain tumors vary as they have irregular form,structure,and volume.The suggested model comprises of three major steps:first,pre-processing to separate the brain from the head and improves contrast stretching.Secondly,P-ABCC is used to obtain tumor edges that are utilized as an initial MRI sequence contour.The level-set segmentation is then used to detect tumor regions from all volume slices with fewer iterations.Results suggest improved model efficiency compared to state-of-the-art methods for both datasets BRATS 2019 and BRATS 2017.At BRATS 2019,dice progress was achieved for Entire Tumor(WT),Tumor Center(TC),and Improved Tumor(ET)by 0.03%,0.03%,and 0.01%respectively.At BRATS 2017,an increase in precision for WT was reached by 5.27%.

3DMKDR:3D Multiscale Kernels CNN Model for Depression Recognition Based on EEG

Depression has become a major health threat around the world,especially for older people,so the effective detection method for depression is a great public health challenge.Electroencephalogram(EEG)can be used as a biomarker to effectively explore depression recognition.Motivated by the studies that multiple smaller scale kernels could increase nonlinear expression compared to a larger kernel,this article proposes a model named the three-dimensional multiscale kernels convolutional neural network model for the depression disorder recognition(3DMKDR),which is a three-dimensional convolutional neural network model with multiscale convolutional kernels for depression recognition based on EEG signals.A three-dimensional structure of the EEG is built by extending one-dimensional feature sequences into a two-dimensional electrode matrix to excavate the related spatiotemporal information among electrodes and the collected electrode matrix.By the major depressive disorder(MDD)and the multi-modal open dataset for mental-disorder analysis(MODMA)datasets,the experiment shows that the accuracies of depression recognition are up to99.86%and 98.01%in the subject-dependent experiment,and 95.80%and 82.27%in the subjectindependent experiment,which are higher than alternative competitive methods.The experimental results demonstrate that the proposed 3DMKDR is potentially useful for depression recognition in older persons in the future.

多模态磁共振成像联合血清GDF3、HSP90A诊断乳腺癌的临床价值

目的探讨多模态磁共振成像(M R I)联合血清人生长分化因子3(GDF3)、热休克蛋白-90α(HSP90A)诊断乳腺癌(BRCA)的临床价值。方法收集2017年1月-2020年12月间在本院健康检查后怀疑为BRCA的96例乳腺疾病患者作为研究对象。以术后病理或穿刺活检结果为标准,将疑似患者分为BRCA组65例,良性组31例。所有受试者接受多模态MRI检查;酶联免疫吸附法检测血清GDF3、HSP90A水平,ROC和四表格分析多模态MRI、血清GDF3、HSP90A水平单独及联合诊断BRCA的价值。结果BRCA组Ktrans、Kep、MD显著高于良性组,ADCslow、ADCfast、MK均低于良性组(P<0.05);DCE-MRI、IVIM及DKI参数(Kep、ADCslow及MK值)诊断BRCA的AUC分别为0.724、0.730、0.652,DCEMRI+IVIM+DKI的诊断效能高于单一模型(Z=2.287~3.793,P=0.001~0.022),AUC为0.839。BRCA组血清GDF3、HSP90A水平均显著高于良性组(P<0.05);血清GDF3、HSP90A水平诊断BRCA的AUC为0.828、0.817,敏感度、70.77%、66.15%;特异度分别为83.87%、93.55%。多模态MRI联合血清GDF3、HSP90A检出假阳性6例,假阴性6例,Kappa值为0.714(P<0.05),与病理结果一致性较高,联合诊断BRCA的灵敏度、阴性预测值及准确度明显高于多模态MRI、血清GDF3、HSP90A单独诊断(P<0.05)。结论多模态MRI联合血清GDF3、HSP90A水平诊断BRCA具有较高的敏感度和准确度,具有一定临床应用价值。