From text to image:challenges in integrating vision into ChatGPT for medical image interpretation

Large language models(LLMs),such as ChatGPT developed by OpenAI,represent a significant advancement in artificial intelligence(AI),designed to understand,generate,and interpret human language by analyzing extensive text data.Their potential integration into clinical settings offers a promising avenue that could transform clinical diagnosis and decision-making processes in the future(Thirunavukarasu et al.,2023).This article aims to provide an in-depth analysis of LLMs’current and potential impact on clinical practices.Their ability to generate differential diagnosis lists underscores their potential as invaluable tools in medical practice and education(Hirosawa et al.,2023;Koga et al.,2023).

基于手机拍照结合Image J软件对干辣椒外观品质的分级研究

干辣椒外观形状和色泽是其品质分类的重要指标。目前GB 10465—1989《辣椒干》中对干辣椒外观形状和色泽的检测方式还停留在人工检测阶段,通常受到主观感知、误差、视觉生理等多种因素影响,未形成科学标准化的检测方法。该研究利用手机拍照对干辣椒获取图像,通过Image J软件进行图像处理,提出了一种便捷、快速、准确的干辣椒外观形状相关特征量的测定方法。与游标卡尺法、剪纸法等人工测量相比,该方法更方便快速,可用于干辣椒的长度、宽度、面积等表型指标的测量。同时,通过构建红绿蓝(RGB)色彩模型获得干辣椒的外观颜色特征参数,色泽分选采用R/(G+B)比率为分级依据,结合干辣椒宽长比和面积可以将干辣椒分为优质、合格、不合格3个等级。

Congruent Feature Selection Method to Improve the Efficacy of Machine Learning-Based Classification in Medical Image Processing

Machine learning(ML)is increasingly applied for medical image processing with appropriate learning paradigms.These applications include analyzing images of various organs,such as the brain,lung,eye,etc.,to identify specific flaws/diseases for diagnosis.The primary concern of ML applications is the precise selection of flexible image features for pattern detection and region classification.Most of the extracted image features are irrelevant and lead to an increase in computation time.Therefore,this article uses an analytical learning paradigm to design a Congruent Feature Selection Method to select the most relevant image features.This process trains the learning paradigm using similarity and correlation-based features over different textural intensities and pixel distributions.The similarity between the pixels over the various distribution patterns with high indexes is recommended for disease diagnosis.Later,the correlation based on intensity and distribution is analyzed to improve the feature selection congruency.Therefore,the more congruent pixels are sorted in the descending order of the selection,which identifies better regions than the distribution.Now,the learning paradigm is trained using intensity and region-based similarity to maximize the chances of selection.Therefore,the probability of feature selection,regardless of the textures and medical image patterns,is improved.This process enhances the performance of ML applications for different medical image processing.The proposed method improves the accuracy,precision,and training rate by 13.19%,10.69%,and 11.06%,respectively,compared to other models for the selected dataset.The mean error and selection time is also reduced by 12.56%and 13.56%,respectively,compared to the same models and dataset.

Semantic Segmentation of Lumbar Vertebrae Using Meijering U-Net(MU-Net)on Spine Magnetic Resonance Images

Lower back pain is one of the most common medical problems in the world and it is experienced by a huge percentage of people everywhere.Due to its ability to produce a detailed view of the soft tissues,including the spinal cord,nerves,intervertebral discs,and vertebrae,Magnetic Resonance Imaging is thought to be the most effective method for imaging the spine.The semantic segmentation of vertebrae plays a major role in the diagnostic process of lumbar diseases.It is difficult to semantically partition the vertebrae in Magnetic Resonance Images from the surrounding variety of tissues,including muscles,ligaments,and intervertebral discs.U-Net is a powerful deep-learning architecture to handle the challenges of medical image analysis tasks and achieves high segmentation accuracy.This work proposes a modified U-Net architecture namely MU-Net,consisting of the Meijering convolutional layer that incorporates the Meijering filter to perform the semantic segmentation of lumbar vertebrae L1 to L5 and sacral vertebra S1.Pseudo-colour mask images were generated and used as ground truth for training the model.The work has been carried out on 1312 images expanded from T1-weighted mid-sagittal MRI images of 515 patients in the Lumbar Spine MRI Dataset publicly available from Mendeley Data.The proposed MU-Net model for the semantic segmentation of the lumbar vertebrae gives better performance with 98.79%of pixel accuracy(PA),98.66%of dice similarity coefficient(DSC),97.36%of Jaccard coefficient,and 92.55%mean Intersection over Union(mean IoU)metrics using the mentioned dataset.

A novel detection method for warhead fragment targets in optical images under dynamic strong interference environments

A measurement system for the scattering characteristics of warhead fragments based on high-speed imaging systems offers advantages such as simple deployment,flexible maneuverability,and high spatiotemporal resolution,enabling the acquisition of full-process data of the fragment scattering process.However,mismatches between camera frame rates and target velocities can lead to long motion blur tails of high-speed fragment targets,resulting in low signal-to-noise ratios and rendering conventional detection algorithms ineffective in dynamic strong interference testing environments.In this study,we propose a detection framework centered on dynamic strong interference disturbance signal separation and suppression.We introduce a mixture Gaussian model constrained under a joint spatialtemporal-transform domain Dirichlet process,combined with total variation regularization to achieve disturbance signal suppression.Experimental results demonstrate that the proposed disturbance suppression method can be integrated with certain conventional motion target detection tasks,enabling adaptation to real-world data to a certain extent.Moreover,we provide a specific implementation of this process,which achieves a detection rate close to 100%with an approximate 0%false alarm rate in multiple sets of real target field test data.This research effectively advances the development of the field of damage parameter testing.

An intelligent coastline interpretation of several types of seacoasts from TM/ETM+ images based on rules

A coastline is defined as the average spring tide line. Different types of seacoast, such as sandy, silty, and bio- logical coast, have different indicators of interpretation. It is very difficult to develop a universal method for interpreting all shorelines. Therefore, the sandy, the silty, and the biological coast are regarded as research objects, and with data mining technolog,found the rules of interpretation of those three types of coastlines. Then, an intelligent coastline interpretation method based on rules was proposed. Firstly, the rules for ex- tracting the waterline in Landsat TM/ETM＋ （Thematic Mapper/Enhanced Thematic Mapper Plus） imagery were discovered. Then, through analyzing the features of sandy, silty and biological coast, the indicators of interpreting different types of shoreline were determined. According to the indicators, the waterline could be corrected to the real coastline. In order to verify the validity of the proposed algorithms, three Landsat TM/ETM＋ imageries were selected for case studies. The experimental results showed that the proposed methods could interpret the coastlines of sandy; silty, and biological coasts with high precision and without human intervention, which exceeded three pixels.

NEW SAR IMAGE INTERPRETATION METHOD OF AIRCRAFT BASED ON JOINT TIME-FREQUENCY ANALYSIS

With the continuous improvement of Synthetic Aperture Radar(SAR) resolution, interpreting the small targets like aircraft in SAR images becomes possible and turn out to be a hot spot in SAR application research. However, due to the complexity of SAR imaging mechanism, interpreting targets in SAR images is a tough problem. This paper presents a new aircraft interpretation method based on the joint time-frequency analysis and multi-dimensional contrasting of basic structures. Moreover, SAR data acquisition experiment is designed for interpreting the aircraft. Analyzing the experiment data with our method, the result shows that the proposed method largely makes use of the SAR data information. The reasonable results can provide some auxiliary support for the SAR images manual interpretation.

Structural Interpretation of Lineaments Uses Satellite Images Processing: A Case Study in North-Eastern Tunisia

This paper presents a lineament detection method using multi-band remote sensing images. The main objective of this work is to design an automatic image processing tool for lineament mapping from Landsat-7 ETM + satellite data. Five procedures were involved: 1) The Principal Component Analysis;2) image enhancement using histogram equalization technique 3) directional Sobel filters of the original data;4) histogram segmentation and 5) binary image generation. The applied methodology was contributed in identifying several known large-scale faults in the Northeast of Tunisia. The statistical and spatial analyses of lineament map indicate a difference of morphological appearance of lineaments in the satellite image. Indeed, all the lineaments present a specific organization. Five groups were classified based on three orientations: NE-SW, E-W and NW-SE. The overlapping of lineament map with the geologic map confirms that these lineaments of diverse directions can be identified and recognized on the field as a fault. The identified lineaments were linked to a deep faults caused by tectonic movements in Tunisia. This study shows the performance of the satellite image processing in the analysis and mapping of the accidents in the northern Atlas.

Shear-Flow Induced Secondary Circulation in Parallel Underwater Topographic Corrugation and Its Application to Satellite Image Interpretation

This study aims to figure out satellite imaging mechanisms for submerged sand ridges in the shallow water region in the case of the flow parallel to the topography corrugation.Solving the disturbance governing equations of the shear-flow yields the analytical solutions of the secondary circulation.The solutions indicate that a flow with a parabolic horizontal velocity shear and a sinusoidal vertical velocity shear will induce a pair of vortexes with opposite signs distributed symmetrically on the two sides of central line of a rectangular canal.In the case of the presence of surface Ekman layer with the direction of Ekman current opposite to(coincident with) the mean flow,the two vortexes converge(diverge) at the central line of canal in the upper layer and form a surface current convergent(divergent) zone along the central line of the canal.In the case of the absence of surface Ekman layer,there is no convergent(divergent) zone formed over the sea surface.The theoretical results are applied to interpretations of three convergent cases,one divergent case and statistics of 27 cases of satellite observations in the submerged sand ridge region of the Liaodong Shoal in the Bohai Sea.We found that the long,finger-like,bright patterns on SAR images are corresponding to the locations of the canals(or tidal channels) formed by two adjacent sand ridges rather than the sand ridges themselves.

Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning

Deep learning (DL) has seen an exponential development in recent years, with major impact in many medical fields, especially in the field of medical image. The purpose of the work converges in determining the importance of each component, describing the specificity and correlations of these elements involved in achieving the precision of interpretation of medical images using DL. The major contribution of this work is primarily to the updated characterisation of the characteristics of the constituent elements of the deep learning process, scientific data, methods of knowledge incorporation, DL models according to the objectives for which they were designed and the presentation of medical applications in accordance with these tasks. Secondly, it describes the specific correlations between the quality, type and volume of data, the deep learning patterns used in the interpretation of diagnostic medical images and their applications in medicine. Finally presents problems and directions of future research. Data quality and volume, annotations and labels, identification and automatic extraction of specific medical terms can help deep learning models perform image analysis tasks. Moreover, the development of models capable of extracting unattended features and easily incorporated into the architecture of DL networks and the development of techniques to search for a certain network architecture according to the objectives set lead to performance in the interpretation of medical images.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

Dendritic Learning-Incorporated Vision Transformer for Image Recognition

Dear Editor,This letter proposes to integrate dendritic learnable network architecture with Vision Transformer to improve the accuracy of image recognition.In this study,based on the theory of dendritic neurons in neuroscience,we design a network that is more practical for engineering to classify visual features.Based on this,we propose a dendritic learning-incorporated vision Transformer(DVT),which out-performs other state-of-the-art methods on three image recognition benchmarks.

Using restored two-dimensional X-ray images to reconstruct the three-dimensional magnetopause

Astronomical imaging technologies are basic tools for the exploration of the universe,providing basic data for the research of astronomy and space physics.The Soft X-ray Imager(SXI)carried by the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)aims to capture two-dimensional(2-D)images of the Earth’s magnetosheath by using soft X-ray imaging.However,the observed 2-D images are affected by many noise factors,destroying the contained information,which is not conducive to the subsequent reconstruction of the three-dimensional(3-D)structure of the magnetopause.The analysis of SXI-simulated observation images shows that such damage cannot be evaluated with traditional restoration models.This makes it difficult to establish the mapping relationship between SXIsimulated observation images and target images by using mathematical models.We propose an image restoration algorithm for SXIsimulated observation images that can recover large-scale structure information on the magnetosphere.The idea is to train a patch estimator by selecting noise–clean patch pairs with the same distribution through the Classification–Expectation Maximization algorithm to achieve the restoration estimation of the SXI-simulated observation image,whose mapping relationship with the target image is established by the patch estimator.The Classification–Expectation Maximization algorithm is used to select multiple patch clusters with the same distribution and then train different patch estimators so as to improve the accuracy of the estimator.Experimental results showed that our image restoration algorithm is superior to other classical image restoration algorithms in the SXI-simulated observation image restoration task,according to the peak signal-to-noise ratio and structural similarity.The restoration results of SXI-simulated observation images are used in the tangent fitting approach and the computed tomography approach toward magnetospheric reconstruction techniques,significantly improving the reconstruction results.Hence,the proposed technology may be feasible for processing SXI-simulated observation images.

Classification and detection of dental images using meta-learning

Meta-learning of dental X-rays is a machine learning technique that can be used to train models to perform new tasks quickly and with minimal input.Instead of just memorizing a task,this is accomplished through teaching a model how to learn.Algorithms for meta-learning are typically trained on a collection of training problems,each of which has a limited number of labelled instances.Multiple Xray classification tasks,including the detection of pneumonia,coronavirus disease 2019,and other disorders,have demonstrated the effectiveness of meta-learning.Meta-learning has the benefit of allowing models to be trained on dental X-ray datasets that are too few for more conventional machine learning methods.Due to the high cost and lengthy collection process associated with dental imaging datasets,this is significant for dental X-ray classification jobs.The ability to train models that are more resistant to fresh input is another benefit of meta-learning.

The Soft X-ray Imager(SXI)on the SMILE Mission

The Soft X-ray Imager(SXI)is part of the scientific payload of the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.SMILE is a joint science mission between the European Space Agency(ESA)and the Chinese Academy of Sciences(CAS)and is due for launch in 2025.SXI is a compact X-ray telescope with a wide field-of-view(FOV)capable of encompassing large portions of Earth’s magnetosphere from the vantage point of the SMILE orbit.SXI is sensitive to the soft X-rays produced by the Solar Wind Charge eXchange(SWCX)process produced when heavy ions of solar wind origin interact with neutral particles in Earth’s exosphere.SWCX provides a mechanism for boundary detection within the magnetosphere,such as the position of Earth’s magnetopause,because the solar wind heavy ions have a very low density in regions of closed magnetic field lines.The sensitivity of the SXI is such that it can potentially track movements of the magnetopause on timescales of a few minutes and the orbit of SMILE will enable such movements to be tracked for segments lasting many hours.SXI is led by the University of Leicester in the United Kingdom(UK)with collaborating organisations on hardware,software and science support within the UK,Europe,China and the United States.

Background removal from global auroral images:Data-driven dayglow modeling

Global images of auroras obtained by cameras on spacecraft are a key tool for studying the near-Earth environment.However,the cameras are sensitive not only to auroral emissions produced by precipitating particles,but also to dayglow emissions produced by photoelectrons induced by sunlight.Nightglow emissions and scattered sunlight can contribute to the background signal.To fully utilize such images in space science,background contamination must be removed to isolate the auroral signal.Here we outline a data-driven approach to modeling the background intensity in multiple images by formulating linear inverse problems based on B-splines and spherical harmonics.The approach is robust,flexible,and iteratively deselects outliers,such as auroral emissions.The final model is smooth across the terminator and accounts for slow temporal variations and large-scale asymmetries in the dayglow.We demonstrate the model by using the three far ultraviolet cameras on the Imager for Magnetopause-to-Aurora Global Exploration(IMAGE)mission.The method can be applied to historical missions and is relevant for upcoming missions,such as the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.

An improved deep dilated convolutional neural network for seismic facies interpretation

With the successful application and breakthrough of deep learning technology in image segmentation,there has been continuous development in the field of seismic facies interpretation using convolutional neural networks.These intelligent and automated methods significantly reduce manual labor,particularly in the laborious task of manually labeling seismic facies.However,the extensive demand for training data imposes limitations on their wider application.To overcome this challenge,we adopt the UNet architecture as the foundational network structure for seismic facies classification,which has demonstrated effective segmentation results even with small-sample training data.Additionally,we integrate spatial pyramid pooling and dilated convolution modules into the network architecture to enhance the perception of spatial information across a broader range.The seismic facies classification test on the public data from the F3 block verifies the superior performance of our proposed improved network structure in delineating seismic facies boundaries.Comparative analysis against the traditional UNet model reveals that our method achieves more accurate predictive classification results,as evidenced by various evaluation metrics for image segmentation.Obviously,the classification accuracy reaches an impressive 96%.Furthermore,the results of seismic facies classification in the seismic slice dimension provide further confirmation of the superior performance of our proposed method,which accurately defines the range of different seismic facies.This approach holds significant potential for analyzing geological patterns and extracting valuable depositional information.

Method for evaluation of geological strength index of carbonate cliff rocks:Coupled hyperspectral-digital borehole image technique

The deterioration of unstable rock mass raised interest in evaluating rock mass quality.However,the traditional evaluation method for the geological strength index(GSI)primarily emphasizes the rock structure and characteristics of discontinuities.It ignores the influence of mineral composition and shows a deficiency in assessing the integrity coefficient.In this context,hyperspectral imaging and digital panoramic borehole camera technologies are applied to analyze the mineral content and integrity of rock mass.Based on the carbonate mineral content and fissure area ratio,the strength reduction factor and integrity coefficient are calculated to improve the GSI evaluation method.According to the results of mineral classification and fissure identification,the strength reduction factor and integrity coefficient increase with the depth of rock mass.The rock mass GSI calculated by the improved method is mainly concentrated between 40 and 60,which is close to the calculation results of the traditional method.The GSI error rates obtained by the two methods are mostly less than 10%,indicating the rationality of the hyperspectral-digital borehole image coupled evaluation method.Moreover,the sensitivity of the fissure area ratio(Sr)to GSI is greater than that of the strength reduction factor(a),which means the proposed GSI is suitable for rocks with significant fissure development.The improved method reduces the influence of subjective factors and provides a reliable index for the deterioration evaluation of rock mass.

Structural Interpretation of Lineaments by Satellite Image Processing(Landsat TM)in the Region of Zahret Medien(Northern Tunisia)

There are several techniques that were developed for determining the linear features. Lineament extraction?from satellite data has been the most widely used applications in geology. In the present study, lineament has?been extracted from the digital satellite scene (Landsat 5, TM data), in the region of Zahret Median situated in the north west of Tunisia. The image was enhanced and used for automatic extraction. Several directions of features were mapped. The directions of major invoices are NE-SW and NW-SE oriented. The validation of the obtained results is carried out by comparison with the results geophysics as well as to the studies previous of mapping developed in the sector of study.

Deep learning-based inpainting of saturation artifacts in optical coherence tomography images

Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.