Importance-aware 3D volume visualization for medical content-based image retrieval-a preliminary study

Background A medical content-based image retrieval(CBIR)system is designed to retrieve images from large imaging repositories that are visually similar to a user′s query image.CBIR is widely used in evidence-based diagnosis,teaching,and research.Although the retrieval accuracy has largely improved,there has been limited development toward visualizing important image features that indicate the similarity of retrieved images.Despite the prevalence of 3D volumetric data in medical imaging such as computed tomography(CT),current CBIR systems still rely on 2D cross-sectional views for the visualization of retrieved images.Such 2D visualization requires users to browse through the image stacks to confirm the similarity of the retrieved images and often involves mental reconstruction of 3D information,including the size,shape,and spatial relations of multiple structures.This process is time-consuming and reliant on users'experience.Methods In this study,we proposed an importance-aware 3D volume visualization method.The rendering parameters were automatically optimized to maximize the visibility of important structures that were detected and prioritized in the retrieval process.We then integrated the proposed visualization into a CBIR system,thereby complementing the 2D cross-sectional views for relevance feedback and further analyses.Results Our preliminary results demonstrate that 3D visualization can provide additional information using multimodal positron emission tomography and computed tomography(PETCT)images of a non-small cell lung cancer dataset.

Evaluating the use of three-dimensional reconstruction visualization technology for precise laparoscopic resection in gastroesophageal junction cancer

BACKGROUND Laparoscopic gastrectomy for esophagogastric junction(EGJ)carcinoma enables the removal of the carcinoma at the junction between the stomach and esophagus while preserving the gastric function,thereby providing patients with better treatment outcomes and quality of life.Nonetheless,this surgical technique also presents some challenges and limitations.Therefore,three-dimensional reconstruction visualization technology(3D RVT)has been introduced into the procedure,providing doctors with more comprehensive and intuitive anatomical information that helps with surgical planning,navigation,and outcome evaluation.AIM To discuss the application and advantages of 3D RVT in precise laparoscopic resection of EGJ carcinomas.METHODS Data were obtained from the electronic or paper-based medical records at The First Affiliated Hospital of Hebei North University from January 2020 to June 2022.A total of 120 patients diagnosed with EGJ carcinoma were included in the study.Of these,68 underwent laparoscopic resection after computed tomography(CT)-enhanced scanning and were categorized into the 2D group,whereas 52 underwent laparoscopic resection after CT-enhanced scanning and 3D RVT and were categorized into the 3D group.This study had two outcome measures:the deviation between tumor-related factors(such as maximum tumor diameter and infiltration length)in 3D RVT and clinical reality,and surgical outcome indicators(such as operative time,intraoperative blood loss,number of lymph node dissections,R0 resection rate,postoperative hospital stay,postoperative gas discharge time,drainage tube removal time,and related complications)between the 2D and 3D groups.RESULTS Among patients included in the 3D group,27 had a maximum tumor diameter of less than 3 cm,whereas 25 had a diameter of 3 cm or more.In actual surgical observations,24 had a diameter of less than 3 cm,whereas 28 had a diameter of 3 cm or more.The findings were consistent between the two methods(χ^(2)=0.346,P=0.556),with a kappa consistency coefficient of 0.808.With respect to infiltration length,in the 3D group,23 patients had a length of less than 5 cm,whereas 29 had a length of 5 cm or more.In actual surgical observations,20 cases had a length of less than 5 cm,whereas 32 had a length of 5 cm or more.The findings were consistent between the two methods(χ^(2)=0.357,P=0.550),with a kappa consistency coefficient of 0.486.Pearson correlation analysis showed that the maximum tumor diameter and infiltration length measured using 3D RVT were positively correlated with clinical observations during surgery(r=0.814 and 0.490,both P<0.05).The 3D group had a shorter operative time(157.02±8.38 vs 183.16±23.87),less intraoperative blood loss(83.65±14.22 vs 110.94±22.05),and higher number of lymph node dissections(28.98±2.82 vs 23.56±2.77)and R0 resection rate(80.77%vs 61.64%)than the 2D group.Furthermore,the 3D group had shorter hospital stay[8(8,9)vs 13(14,16)],time to gas passage[3(3,4)vs 4(5,5)],and drainage tube removal time[4(4,5)vs 6(6,7)]than the 2D group.The complication rate was lower in the 3D group(11.54%)than in the 2D group(26.47%)(χ^(2)=4.106,P<0.05).CONCLUSION Using 3D RVT,doctors can gain a more comprehensive and intuitive understanding of the anatomy and related lesions of EGJ carcinomas,thus enabling more accurate surgical planning.

Painting image browser applying an associate-rule-aware multidimensional data visualization technique

Exploration of artworks is enjoyable but often time consuming.For example,it is not always easy to discover the favorite types of unknown painting works.It is not also always easy to explore unpopular painting works which looks similar to painting works created by famous artists.This paper presents a painting image browser which assists the explorative discovery of user-interested painting works.The presented browser applies a new multidimensional data visualization technique that highlights particular ranges of particular numeric values based on association rules to suggest cues to find favorite painting images.This study assumes a large number of painting images are provided where categorical information(e.g.,names of artists,created year)is assigned to the images.The presented system firstly calculates the feature values of the images as a preprocessing step.Then the browser visualizes the multidimensional feature values as a heatmap and highlights association rules discovered from the relationships between the feature values and categorical information.This mechanism enables users to explore favorite painting images or painting images that look similar to famous painting works.Our case study and user evaluation demonstrates the effectiveness of the presented image browser.

In situ nanobubble sizing by visualization particle tracking and image-based dynamic light scattering

A novel method combining visualization particle tracking with image-based dynamic light scattering was developed to achieve the in situ and real-time size measurement of nanobubbles(NBs).First,the in situ size distribution of NBs was visualized by dark-field microscopy.Then,real-time size during the preparation was measured using image-based dynamic light scattering,and the longitudinal size distribution of NBs in the sample cell was obtained in a steady state.Results show that this strategy can provide a detailed and accurate size of bubbles in the whole sample compared with the commercial ZetaSizer Nano equipment.Therefore,the developed method is a real-time and simple technology with excellent accuracy,providing new insights into the accurate measurement of the size distribution of NBs or nanoparticles in solution.

Explosive Field Visualization Based on Image Fusion

The advantages and disadvantages of two existing methods for explosive field visualization are analyzed in this paper. And a new method based on image fusion is proposed to integrate their complementary advantages. With the method, two source images built by equal mapping and modulus mapping are individually decomposed into two Gauss-Laplacian pyramid sequences. Then, the two individual sequences are used to make a composite one according to the process of fusion. Finally, a new image is reconstructed from the composite sequence. Experimental results show that the new images integrate the advantages of sources, effectively improve the visualization, and disclose more information about explosive field.

Visualization of Three-dimensional Human Data Based on CT Image

Three-dimensional medical image visualization becomes an essential part for medical field, including computer aided diagnosis, surgery planning and simulation, artificial limb surgery, radiotherapy planning, and teaching etc. In this paper, marching cubes algorithm is adopted to reconstruct the 3-D images for the CT image sequence in DICOM format under theVC＋＋6.0 and the visual package VTK platform. The relatively simple interactive operations such as rotation and transfer can be realized on the platform. Moreover, the normal vector and interior point are calculated to form the virtual clipping plane, which is then used to incise the 3-D object. Information of the virtual slice can be obtained, in the mean while the virtual slice images are displayed on the screen. The technique can realize the real time interaction extraction of virtual slice on 3-D CT image. The cuboids structured can be zoomed, moved and eircumrotated by operating mouse to incise the 3-D reconstruction object. Real time interaction can be realized by clipping the reconstruction object. The coordinates can be acquired by the mouse clicking in the 3D space, to realize the point mouse pick-up as well angle and distance interactive measurement. We can get quantitative information about 3-D images through measurement.

3D graphical visualization of the genetic architectures underlying complex traits in multiple environments

An approach for generating interactive 3D graphical visualization of the genetic architectures of complex traits in multiple environments is described. 3D graphical visualization is utilized for making improvements on traditional plots in quan- titative trait locus (QTL) mapping analysis. Interactive 3D graphical visualization for abstract expression of QTL, epistasis and their environmental interactions for experimental populations was developed in framework of user-friendly software QTLNetwork (http://ibi.zju.edu.cn/software/qtlnetwork). Novel definition of graphical meta system and computation of virtual coordinates are used to achieve explicit but meaningful visualization. Interactive 3D graphical visualization for QTL analysis provides geneticists and breeders a powerful and easy-to-use tool to analyze and publish their research results.

3-D Visualization of Medical Images with Arbitrary Sections

In this paper, with the general retrospect to the research on surface reconstruction and the marching cubes algorithm, we gave detailed description of an algorithm on the construction of object surfaces. The possible ambiguity problem in the original marching cubes algorithm was eliminated by its index mechanism. Some results on the MRI images were presented. Based on extracting and clipping contours from a set of medial slice images and setting the patch vertices values according to the gray images, this algorithm may be applied to form the arbitrary section images with three dimensional effects. It can also enhance the visual effect and interpretation of medical data.

An Interpretable CNN for the Segmentation of the Left Ventricle in Cardiac MRI by Real-Time Visualization

The interpretability of deep learning models has emerged as a compelling area in artificial intelligence research.The safety criteria for medical imaging are highly stringent,and models are required for an explanation.However,existing convolutional neural network solutions for left ventricular segmentation are viewed in terms of inputs and outputs.Thus,the interpretability of CNNs has come into the spotlight.Since medical imaging data are limited,many methods to fine-tune medical imaging models that are popular in transfer models have been built using massive public Image Net datasets by the transfer learning method.Unfortunately,this generates many unreliable parameters and makes it difficult to generate plausible explanations from these models.In this study,we trained from scratch rather than relying on transfer learning,creating a novel interpretable approach for autonomously segmenting the left ventricle with a cardiac MRI.Our enhanced GPU training system implemented interpretable global average pooling for graphics using deep learning.The deep learning tasks were simplified.Simplification included data management,neural network architecture,and training.Our system monitored and analyzed the gradient changes of different layers with dynamic visualizations in real-time and selected the optimal deployment model.Our results demonstrated that the proposed method was feasible and efficient:the Dice coefficient reached 94.48%,and the accuracy reached 99.7%.It was found that no current transfer learning models could perform comparably to the ImageNet transfer learning architectures.This model is lightweight and more convenient to deploy on mobile devices than transfer learning models.

Research and Application of Log Defect Detection and Visualization System Based on Dry Coupling Ultrasonic Method

In order to optimize the wood internal quality detection and evaluation system and improve the comprehensive utilization rate of wood,this paper invented a set of log internal defect detection and visualization system by using the ultrasonic dry coupling agent method.The detection and visualization analysis of internal log defects were realized through log specimen test.The main conclusions show that the accuracy,reliability and practicability of the system for detecting the internal defects of log specimens have been effectively verified.The system can make the edge of the detected image smooth by interpolation algorithm,and the edge detection algorithm can be used to detect and reflect the location of internal defects of logs accurately.The content mentioned above has good application value for meeting the requirement of increasing demand for wood resources and improving the automation level of wood nondestructive testing instruments.

A visualization pipeline for in vivo two-photon volumetric astrocytic calcium imaging

Astrocytes,the multi-functional glial cells with the most abundant population in the brain,integrate information across their territories to regulate neuronal synaptic and cerebrovascular activities.Astrocytic calcium(Ca^(2+))signaling is the major readout of cellular functional state of astrocytes.The conventional two-photon in vivo imaging usually focuses on a single horizontal focal plane to capture the astrocytic Ca^(2+)signals,which leaves>80%spatial information undetected.To fully probe the Ca^(2+)activity across the whole astrocytic territory,we developed a pipeline for imaging and visualizing volumetric astrocytic Ca^(2+)time-lapse images.With the pipeline,we discovered a new signal distribution pattern from three-dimensional(3D)astrocytic Ca^(2+)imaging data of mice under isoflurane anesthetic states.The tools developed in this study enable a better understanding of the spatiotemporal patterns of astrocytic activity in 3D space.

Machine learning based fileless malware traffic classification using image visualization

In today's interconnected world,network traffic is replete with adversarial attacks.As technology evolves,these attacks are also becoming increasingly sophisticated,making them even harder to detect.Fortunately,artificial intelli-gence(Al)and,specifically machine learning(ML),have shown great success in fast and accurate detection,classifica-tion,and even analysis of such threats.Accordingly,there is a growing body of literature addressing how subfields of Al/ML(e.g.,natural language processing(NLP))are getting leveraged to accurately detect evasive malicious patterns in network traffic.In this paper,we delve into the current advancements in ML-based network traffic classification using image visualization.Through a rigorous experimental methodology,we first explore the process of network traffic to image conversion.Subsequently,we investigate how machine learning techniques can effectively leverage image visualization to accurately classify evasive malicious traces within network traffic.Through the utilization of production-level tools and utilities in realistic experiments,our proposed solution achieves an impressive accuracy rate of 99.48%in detecting fileless malware,which is widely regarded as one of the most elusive classes of malicious software.

3D visualization of the material flow in friction stir welding process

The material flow in friction stir welded 2014 Al alloy has been investigated using a marker insert technique （MIT）. Results of the flow visualization show that the material flow is asymmetrical during the friction stir welding （FSW） process and there are also significant differences in the flow patterns observed on advancing side and retreating side. On advancing side, some material transport forward and some move backward, but on retreating side, material only transport backward. At the top surface of the weld, significant material transport forward due to the action of the rotating tool shoulder. Combining the data from all the markers, a three-dituensional flow visualization, similar to the 3D image reconstruction technique, was obtained. The three-dimensional plot gives the tendency chart of material flow in friction stir welding process and from the plot it can be seen that there is a vertical, circular motion around the longitudinal axis of the weld. On the advancing side of the weld, the material is pushed downward but on the retreating side, the material is pushed toward the crown of the weld. The net result of the two relative motions in both side of the advancing and the retreating is that a circular motion comes into being. Comparatively, the material flow around the longitudinal axis is a secondary motion.

基于Visualization Toolkit的脑模型三维重建方法研究

目的利用可视化工具包VisualizationToolkit(VTK)结合VC++实现医学图像三维可视化。方法基于头部CT测量数据,采用MarchingCubes算法和Raycasting算法重建出头模型的表皮和颅骨。结果和结论VTK使用灵活,功能强大,利用它进行图像重建,具有重建步骤简单、效果好、速度快、交互能力强等优点,可以被广泛应用于医学图像的重建中。

Examining data visualization pitfalls in scientific publications

Data visualization blends art and science to convey stories from data via graphical representations.Considering different problems,applications,requirements,and design goals,it is challenging to combine these two components at their full force.While the art component involves creating visually appealing and easily interpreted graphics for users,the science component requires accurate representations of a large amount of input data.With a lack of the science component,visualization cannot serve its role of creating correct representations of the actual data,thus leading to wrong perception,interpretation,and decision.It might be even worse if incorrect visual representations were intentionally produced to deceive the viewers.To address common pitfalls in graphical representations,this paper focuses on identifying and understanding the root causes of misinformation in graphical representations.We reviewed the misleading data visualization examples in the scientific publications collected from indexing databases and then projected them onto the fundamental units of visual communication such as color,shape,size,and spatial orientation.Moreover,a text mining technique was applied to extract practical insights from common visualization pitfalls.Cochran’s Q test and McNemar’s test were conducted to examine if there is any difference in the proportions of common errors among color,shape,size,and spatial orientation.The findings showed that the pie chart is the most misused graphical representation,and size is the most critical issue.It was also observed that there were statistically significant differences in the proportion of errors among color,shape,size,and spatial orientation.

Simulation and visualization experiment of manganese ion diffusion and damage to gel in a porous media-gel system

A new visualization method for studying the damage to gel structure caused by high salinity ions is explored by using the characteristics of suppression image signal of Mn^(2+) and nuclear magnetic resonance(NMR) imaging technique. The diffusion and distribution characteristics of Mn^(2+) in porous media-gel system were studied based on manganese chloride static diffusion and gel flooding experiments, and the gel's nuclear magnetic image and displacement pressure were tested. The results show that the diffusion of Mn^(2+)conforms to the Fick diffusion law in porous media-gel system, and the diffusion speed of Mn^(2+) increases and the area of gel image decreases gradually with the increase of concentration, and the image of gel decreases faster and the pressure drop of water drive is larger in flooding experiment of manganese chloride with higher concentration. Reaction-diffusion model with the reaction of Mn^(2+) with gel was established to study the concentration distribution characteristics of Mn^(2+). The model is validated by comparing the results with magnetic resonance imaging(MRI) experiments and the diffusion coefficient of Mn^(2+) equals 1.6 mm^2/h, and the minimum concentration of Mn^(2+) to impact gel NMR image signals is 2.5 g/L. The above results show that the diffusion of Mn^(2+) into the gel in the rock core inhibits the imaging signal of the gel and damages its strength, and the greater the concentration is, the greater the influence. Increase of adsorption amount of gel and reaction rate, reduction of diffusion time, and addition of ion adsorption isolator all can reduce the impact of Mn^(2+) on the gel.

Visualization Detection of Solid-Liquid Two-Phase Flow in Filling Pipeline by Electrical Capacitance Tomography Technology

During mine filling,the caking in the pipeline and the waste rock in the filling slurry may cause serious safety accidents such as pipe blocking or explosion.Therefore,the visualization of the innermine filling of the solid-liquid two-phase flow in the pipeline is important.This paper proposes a method based on capacitance tomography for the visualization of the solid-liquid distribution on the section of a filling pipe.A feedback network is used for electrical capacitance tomography reconstruction.This reconstruction method uses radial basis function neural network fitting to determine the relationship between the capacitance vector and medium distribution error.In the reconstruction process,the error in the linear back projection is removed;thus,the reconstruction problem becomes an accurate linear problem.The simulation results showthat the reconstruction accuracy of this algorithm is better than that of many traditional algorithms;furthermore,the reconstructed image artifacts are fewer,and the phase distribution boundary is clearer.This method can help determine the location and size of the caking and waste rock in the cross section of the pipeline more accurately and has great application prospects in the visualization of filling pipelines in mines.

Fast Multilevel CVT-Based Adaptive Data Visualization Algorithm

Efficient data visualization techniques are critical for many scientific applications. Centroidal Voronoi tessellation(CVT) based algorithms offer a convenient vehicle for performing image analysis,segmentation and compression while allowing to optimize retained image quality with respect to a given metric.In experimental science with data counts following Poisson distributions,several CVT-based data tessellation algorithms have been recently developed.Although they surpass their predecessors in robustness and quality of reconstructed data,time consumption remains to be an issue due to heavy utilization of the slowly converging Lloyd iteration.This paper discusses one possible approach to accelerating data visualization algorithms.It relies on a multidimensional generalization of the optimization based multilevel algorithm for the numerical computation of the CVTs introduced in[1],where a rigorous proof of its uniform convergence has been presented in 1-dimensional setting.The multidimensional implementation employs barycentric coordinate based interpolation and maximal independent set coarsening procedures.It is shown that when coupled with bin accretion algorithm accounting for the discrete nature of the data,the algorithm outperforms Lloyd-based schemes and preserves uniform convergence with respect to the problem size.Although numerical demonstrations provided are limited to spectroscopy data analysis,the method has a context-independent setup and can potentially deliver significant speedup to other scientific and engineering applications.

3D Urban Visualization with LOD Techniques

In 3D urban visualization, large data volumes related to buildings are a major factor that limits the delivery and browsing speed in a web-based computer system. This paper proposes a new approach based on the level of detail （LOD） technique advanced in 3D visualization in computer graphics. The key idea of LOD technique is to generalize details of object surfaces without losing details for delivery and displaying objects. This technique has been successfully used in visualizing one or a few multiple objects in films and other industries. However, applying the technique to 3D urban visualization requires an effective generalization method for urban buildings. Conventional two-dimensional （2D） generalization method at different scales provides a good generalization reference for 3D urban visualization, Yet, it is difficult to determine when and where to retrieve data for displaying buildings. To solve this problem, this paper defines an imaging scale point and image scale region for judging when and where to get the fight data for visualization. The results show that the average response time of view transformations is much decreased.

Research of the Visualization Temperature Field of the Communication Room Based on the Reconstruction of Three- dimensional Temperature Field

Whether the temperature of the communication room is high or not is the precondition of the fact that the servers in the cabinets work normally.The traditional method to control temperature is"equilibrium refrigeration",which is high energy consumption and low efficiency.In order to solve such problems,the visualization system of the three-dimensional temperature field of the communication room based on calculation of the reconstruction of the three-dimensional temperature field by collecting the temperatures of the discrete points and on using specific data fitting algorithm is presented.The system uses Direct3D graphics library and designs under the frame of VB.NET,which could monitor the temperature of the communication room in real time and provide visual basis for the effective thermal management in the communication room.