From text to image:challenges in integrating vision into ChatGPT for medical image interpretation

Large language models(LLMs),such as ChatGPT developed by OpenAI,represent a significant advancement in artificial intelligence(AI),designed to understand,generate,and interpret human language by analyzing extensive text data.Their potential integration into clinical settings offers a promising avenue that could transform clinical diagnosis and decision-making processes in the future(Thirunavukarasu et al.,2023).This article aims to provide an in-depth analysis of LLMs’current and potential impact on clinical practices.Their ability to generate differential diagnosis lists underscores their potential as invaluable tools in medical practice and education(Hirosawa et al.,2023;Koga et al.,2023).

Efficient Quadtree based Fractal Image Coding Scheme in Wavelet Transform Domain *

This paper presents an efficient quadtree based fractal image coding scheme in wavelet transform domain based on the wavelet based theory of fractal image compression introduced by Davis. In the scheme, zerotrees of wavelet coefficients are used to reduce the number of domain blocks, which leads to lower bit cost required to represent the location information of fractal coding, and overall entropy constrained optimization is performed for the decision trees as well as for the sets of scalar quantizers and self quantizers of wavelet subtrees. Experiment results show that at the low bit rates, the proposed scheme gives about 1 dB improvement in PSNR over the reported results.

Remote Sensing Image Classification Based on Decision Tree in the Karst Rocky Desertification Areas: A Case Study of Kaizuo Township

Karst rocky desertification is a phenomenon of land degradation as a result of affection by the interaction of natural and human factors.In the past,in the rocky desertification areas,supervised classification and unsupervised classification are often used to classify the remote sensing image.But they only use pixel brightness characteristics to classify it.So the classification accuracy is low and can not meet the needs of practical application.Decision tree classification is a new technology for remote sensing image classification.In this study,we select the rocky desertification areas Kaizuo Township as a case study,use the ASTER image data,DEM and lithology data,by extracting the normalized difference vegetation index,ratio vegetation index,terrain slope and other data to establish classification rules to build decision trees.In the ENVI software support,we access the classification images.By calculating the classification accuracy and kappa coefficient,we find that better classification results can be obtained,desertification information can be extracted automatically and if more remote sensing image bands used,higher resolution DEM employed and less errors data reduced during processing,classification accuracy can be improve further.

A NEW FRACTAL ZEROTREE CODING FOR WAVELET IMAGE

Based on the mechanisms underlying the performance of fractal and Discrete Wavelet Transform(DWT), one method using fractal-based self-quantization coding way to code different subband coefficients of DWT is presented. Within this method finer coefficients are fractal encoded according to the successive coarser ones. Self-similarities inherent between parent and their children at the same spatial location of the adjacent scales of similar orientation are exploited to predict variation of information across wavelet scales. On the other hand, with respect to Human Visual System(HVS) model, we assign different error thresholds to different decomposition scales, and different shape of range blocks to different orientations of the same scale, by which the perceptually lossless high compression ratio can be achieved and the matching processing can be quickened dramatically.

Quantization-Based Robust Image Watermarking Using the Dual Tree Complex Wavelet Transform

Conventional quantization index modulation (QIM) watermarking uses the fixed quantization step size for the host signal.This scheme is not robust against geometric distortions and may lead to poor fidelity in some areas of content.Thus,we proposed a quantization-based image watermarking in the dual tree complex wavelet domain.We took advantages of the dual tree complex wavelets (perfect reconstruction,approximate shift invariance,and directional selectivity).For the case of watermark detecting,the probability of false alarm and probability of false negative were exploited and verified by simulation.Experimental results demonstrate that the proposed method is robust against JPEG compression,additive white Gaussian noise (AWGN),and some kinds of geometric attacks such as scaling,rotation,etc.

Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification.The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles(UAVs) provides a new research direction for urban tree species classification.We proposed an RGB optical image dataset with 10 urban tree species,termed TCC10,which is a benchmark for tree canopy classification(TCC).TCC10 dataset contains two types of data:tree canopy images with simple backgrounds and those with complex backgrounds.The objective was to examine the possibility of using deep learning methods(AlexNet,VGG-16,and ResNet-50) for individual tree species classification.The results of convolutional neural networks(CNNs) were compared with those of K-nearest neighbor(KNN) and BP neural network.Our results demonstrated:(1) ResNet-50 achieved an overall accuracy(OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16.(2) The classification accuracy of KNN and BP neural network was less than70%,while the accuracy of CNNs was relatively higher.(3)The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds.For the deciduous tree species in TCC10,the classification accuracy of ResNet-50 was higher in summer than that in autumn.Therefore,the deep learning is effective for urban tree species classification using RGB optical images.

Multi-scale attention encoder for street-to-aerial image geo-localization

The goal of street-to-aerial cross-view image geo-localization is to determine the location of the query street-view image by retrieving the aerial-view image from the same place.The drastic viewpoint and appearance gap between the aerial-view and the street-view images brings a huge challenge against this task.In this paper,we propose a novel multiscale attention encoder to capture the multiscale contextual information of the aerial/street-view images.To bridge the domain gap between these two view images,we first use an inverse polar transform to make the street-view images approximately aligned with the aerial-view images.Then,the explored multiscale attention encoder is applied to convert the image into feature representation with the guidance of the learnt multiscale information.Finally,we propose a novel global mining strategy to enable the network to pay more attention to hard negative exemplars.Experiments on standard benchmark datasets show that our approach obtains 81.39%top-1 recall rate on the CVUSA dataset and 71.52%on the CVACT dataset,achieving the state-of-the-art performance and outperforming most of the existing methods significantly.

Rapid visual screening of soft-story buildings from street view images using deep learning classification

Rapid and accurate identification of potential structural deficiencies is a crucial task in evaluating seismic vulnerability of large building inventories in a region. In the case of multi-story structures, abrupt vertical variations of story stiffness are known to significantly increase the likelihood of collapse during moderate or severe earthquakes. Identifying and retrofitting buildings with such irregularities—generally termed as soft-story buildings—is, therefore, vital in earthquake preparedness and loss mitigation efforts. Soft-story building identification through conventional means is a labor-intensive and time-consuming process. In this study, an automated procedure was devised based on deep learning techniques for identifying soft-story buildings from street-view images at a regional scale. A database containing a large number of building images and a semi-automated image labeling approach that effectively annotates new database entries was developed for developing the deep learning model. Extensive computational experiments were carried out to examine the effectiveness of the proposed procedure, and to gain insights into automated soft-story building identification.

An image encryption algorithm based on spatiotemporal chaos and middle order traversal of a binary tree

This paper proposes an image encryption algorithm based on spatiotemporal chaos and middle order traversal of a binary tree.Firstly,other programming software is used to perform the middle order traversal,and the plaintext image is sorted according to the middle order traversal sequence on the permutation.Secondly,the chaotic sequence is generated using the coupled map lattice to set the chaotic interference value.Finally,the XOR operation between the adjacent pixel values of the replacement image is completed to generate the ciphertext matrix.The simulation and experimental results show that the proposed algorithm can resist typical attacks and has good robustness.

Fully Convolutional Networks for Street Furniture Identification in Panorama Images

Panoramic images are widely used in many scenes,especially in virtual reality and street view capture.However,they are new for street furniture identification which is usually based on mobile laser scanning point cloud data or conventional 2D images.This study proposes to perform semantic segmentation on panoramic images and transformed images to separate light poles and traffic signs from background implemented by pre-trained Fully Convolutional Networks(FCN).FCN is the most important model for deep learning applied on semantic segmentation for its end to end training process and pixel-wise prediction.In this study,we use FCN-8s model that pre-trained on cityscape dataset and finetune it by our own data.Then replace cross entropy loss function with focal loss function in the FCN model and train it again to produce the predictions.The results show that in all results from pre-trained model,fine-tuning,and FCN model with focal loss,the light poles and traffic signs are detected well and the transformed images have better performance than panoramic images in the prediction according to the Recall and IoU evaluation.

Dendritic Learning-Incorporated Vision Transformer for Image Recognition

Dear Editor,This letter proposes to integrate dendritic learnable network architecture with Vision Transformer to improve the accuracy of image recognition.In this study,based on the theory of dendritic neurons in neuroscience,we design a network that is more practical for engineering to classify visual features.Based on this,we propose a dendritic learning-incorporated vision Transformer(DVT),which out-performs other state-of-the-art methods on three image recognition benchmarks.

Wavelet Coding of Image Using Quadtree and Vector Entropy Coding

A new efficient method based on Quadtree Representation and Vector Entropy Coding (QRVEC) for encoding the wavelet transform coefficients of images is presented. In addition, how to flexibly control the coder’ s output bit rate is also investigated.

An Efficient Approach for Tree Digital Image Segmentation

This paper proposes an improved method to segment tree image based on color and texture feature and amends the segmented result by mathematical morphology. The crown and trunk of one tree have been successfully segmented and the experimental result is deemed effective. The authors conclude that building a standard data base for a range of species, featuring color and texture is a necessary condition and constitutes the essential groundwork for tree image segmentation in order to insure its quality.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

SIXTEEN TREE METHOD FOR BILEVEL IMAGE DATA COMPRESSION

A sixteen tree method of data compression of bilevel image is described.Thismethod has high efficiency,no information loss during compression,and easy to realize.

High-resolution Remote Sensing Image Segmentation Using Minimum Spanning Tree Tessellation and RHMRF-FCM Algorithm

It is proposed a high resolution remote sensing image segmentation method which combines static minimum spanning tree(MST)tessellation considering shape information and the RHMRF-FCM algorithm.It solves the problems in the traditional pixel-based HMRF-FCM algorithm in which poor noise resistance and low precision segmentation in a complex boundary exist.By using the MST model and shape information,the object boundary and geometrical noise can be expressed and reduced respectively.Firstly,the static MST tessellation is employed for dividing the image domain into some sub-regions corresponding to the components of homogeneous regions needed to be segmented.Secondly,based on the tessellation results,the RHMRF model is built,and regulation terms considering the KL information and the information entropy are introduced into the FCM objective function.Finally,the partial differential method and Lagrange function are employed to calculate the parameters of the fuzzy objective function for obtaining the global optimal segmentation results.To verify the robustness and effectiveness of the proposed algorithm,the experiments are carried out with WorldView-3(WV-3)high resolution image.The results from proposed method with different parameters and comparing methods(multi-resolution method and watershed segmentation method in eCognition software)are analyzed qualitatively and quantitatively.

Using restored two-dimensional X-ray images to reconstruct the three-dimensional magnetopause

Astronomical imaging technologies are basic tools for the exploration of the universe,providing basic data for the research of astronomy and space physics.The Soft X-ray Imager(SXI)carried by the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)aims to capture two-dimensional(2-D)images of the Earth’s magnetosheath by using soft X-ray imaging.However,the observed 2-D images are affected by many noise factors,destroying the contained information,which is not conducive to the subsequent reconstruction of the three-dimensional(3-D)structure of the magnetopause.The analysis of SXI-simulated observation images shows that such damage cannot be evaluated with traditional restoration models.This makes it difficult to establish the mapping relationship between SXIsimulated observation images and target images by using mathematical models.We propose an image restoration algorithm for SXIsimulated observation images that can recover large-scale structure information on the magnetosphere.The idea is to train a patch estimator by selecting noise–clean patch pairs with the same distribution through the Classification–Expectation Maximization algorithm to achieve the restoration estimation of the SXI-simulated observation image,whose mapping relationship with the target image is established by the patch estimator.The Classification–Expectation Maximization algorithm is used to select multiple patch clusters with the same distribution and then train different patch estimators so as to improve the accuracy of the estimator.Experimental results showed that our image restoration algorithm is superior to other classical image restoration algorithms in the SXI-simulated observation image restoration task,according to the peak signal-to-noise ratio and structural similarity.The restoration results of SXI-simulated observation images are used in the tangent fitting approach and the computed tomography approach toward magnetospheric reconstruction techniques,significantly improving the reconstruction results.Hence,the proposed technology may be feasible for processing SXI-simulated observation images.

The Soft X-ray Imager(SXI)on the SMILE Mission

The Soft X-ray Imager(SXI)is part of the scientific payload of the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.SMILE is a joint science mission between the European Space Agency(ESA)and the Chinese Academy of Sciences(CAS)and is due for launch in 2025.SXI is a compact X-ray telescope with a wide field-of-view(FOV)capable of encompassing large portions of Earth’s magnetosphere from the vantage point of the SMILE orbit.SXI is sensitive to the soft X-rays produced by the Solar Wind Charge eXchange(SWCX)process produced when heavy ions of solar wind origin interact with neutral particles in Earth’s exosphere.SWCX provides a mechanism for boundary detection within the magnetosphere,such as the position of Earth’s magnetopause,because the solar wind heavy ions have a very low density in regions of closed magnetic field lines.The sensitivity of the SXI is such that it can potentially track movements of the magnetopause on timescales of a few minutes and the orbit of SMILE will enable such movements to be tracked for segments lasting many hours.SXI is led by the University of Leicester in the United Kingdom(UK)with collaborating organisations on hardware,software and science support within the UK,Europe,China and the United States.

Background removal from global auroral images:Data-driven dayglow modeling

Global images of auroras obtained by cameras on spacecraft are a key tool for studying the near-Earth environment.However,the cameras are sensitive not only to auroral emissions produced by precipitating particles,but also to dayglow emissions produced by photoelectrons induced by sunlight.Nightglow emissions and scattered sunlight can contribute to the background signal.To fully utilize such images in space science,background contamination must be removed to isolate the auroral signal.Here we outline a data-driven approach to modeling the background intensity in multiple images by formulating linear inverse problems based on B-splines and spherical harmonics.The approach is robust,flexible,and iteratively deselects outliers,such as auroral emissions.The final model is smooth across the terminator and accounts for slow temporal variations and large-scale asymmetries in the dayglow.We demonstrate the model by using the three far ultraviolet cameras on the Imager for Magnetopause-to-Aurora Global Exploration(IMAGE)mission.The method can be applied to historical missions and is relevant for upcoming missions,such as the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.

Image Coding Using Wavelet Transform and EntropyConstrained Vector Quantization with Quadtree Structure Vectors

This paper presents a new wavelet transform image coding method. On the basis of a hierarchical wavelet decomposition of images, entropy constrained vector quantization is employed to encode the wavelet coefficients at all the high frequency bands with