Background A medical content-based image retrieval(CBIR)system is designed to retrieve images from large imaging repositories that are visually similar to a user′s query image.CBIR is widely used in evidence-based di...Background A medical content-based image retrieval(CBIR)system is designed to retrieve images from large imaging repositories that are visually similar to a user′s query image.CBIR is widely used in evidence-based diagnosis,teaching,and research.Although the retrieval accuracy has largely improved,there has been limited development toward visualizing important image features that indicate the similarity of retrieved images.Despite the prevalence of 3D volumetric data in medical imaging such as computed tomography(CT),current CBIR systems still rely on 2D cross-sectional views for the visualization of retrieved images.Such 2D visualization requires users to browse through the image stacks to confirm the similarity of the retrieved images and often involves mental reconstruction of 3D information,including the size,shape,and spatial relations of multiple structures.This process is time-consuming and reliant on users'experience.Methods In this study,we proposed an importance-aware 3D volume visualization method.The rendering parameters were automatically optimized to maximize the visibility of important structures that were detected and prioritized in the retrieval process.We then integrated the proposed visualization into a CBIR system,thereby complementing the 2D cross-sectional views for relevance feedback and further analyses.Results Our preliminary results demonstrate that 3D visualization can provide additional information using multimodal positron emission tomography and computed tomography(PETCT)images of a non-small cell lung cancer dataset.展开更多
In order to perform a high-quality interactive rendering of large medical data sets on a single off-the-shelf PC, a LOD selection algorithm for multi-resolution volume rendering using 3D texture mapping is presented, ...In order to perform a high-quality interactive rendering of large medical data sets on a single off-the-shelf PC, a LOD selection algorithm for multi-resolution volume rendering using 3D texture mapping is presented, which uses an adaptive scheme that renders the volume in a region-of-interest at a high resolution and the volume away from this region at lower resolutions. The algorithm is based on several important criteria, and rendering is done adaptively by selecting high-resolution cells close to a center of attention and low-resolution cells away from this area. In addition, our hierarchical level-of-detail representation guarantees consistent interpolation between different resolution levels. Experiments have been applied to a number of large medical data and have produced high quality images at interactive frame rates using standard PC hardware.展开更多
Direct isosurface volume rendering is the most prominent modern method for medical data visualization.It is based on finding intersection points between the rays corresponding to pixels on the screen and isosurface. T...Direct isosurface volume rendering is the most prominent modern method for medical data visualization.It is based on finding intersection points between the rays corresponding to pixels on the screen and isosurface. This article describes a two-pass algorithm for accelerating the method on the graphic processing unit(GPU). On the first pass, the intersections with the isosurface are found only for a small number of rays, which is done by rendering into a lower-resolution texture. On the second pass, the obtained information is used to efficiently calculate the intersection points of all the other. The number of rays to use during the first pass is determined by using an adaptive algorithm, which runs on the central processing unit(CPU) in parallel with the second pass of the rendering. The proposed approach allows to significantly speed up isosurface visualization without quality loss. Experiments show acceleration up to 10 times in comparison with a common ray casting method implemented on GPU. To the authors’ knowledge, this is the fastest approach for ray casting which does not require any preprocessing and could be run on common GPUs.展开更多
Elliptical splats are used to represent and render the isosurface of volume data. The method consists of two steps. The first step is to extract points on the isosurface by looking up the case table. In the second ste...Elliptical splats are used to represent and render the isosurface of volume data. The method consists of two steps. The first step is to extract points on the isosurface by looking up the case table. In the second step, properties of splats are computed based on local geometry. Rendering is achieved using surface splatting algorithm. The obtained results show that the extraction time of isosurfaces can be reduced by a factor of three. So our approach is more appropriate for interactive visualization of large medical data than the classical marching cubes (MC) technique.展开更多
Direct volume rendering (DVR) is a powerful visualization technique which allows users to effectively explore and study volumetric datasets. Different transparency settings can be flexibly assigned to different stru...Direct volume rendering (DVR) is a powerful visualization technique which allows users to effectively explore and study volumetric datasets. Different transparency settings can be flexibly assigned to different structures such that some valuable information can be revealed in direct volume rendered images (DVRIs). However, end-users often feel that some risks are always associated with DVR because they do not know whether any important information is missing from the transparent regions of DVRIs. In this paper, we investigate how to semi-automatically generate a set of DVRIs and also an animation which can reveal information missed in the original DVRIs and meanwhile satisfy some image quality criteria such as coherence. A complete framework is developed to tackle various problems related to the generation and quality evaluation of visibility-aware DVRIs and animations. Our technique can reduce the risk of using direct volume rendering and thus boost the confidence of users in volume rendering systems.展开更多
Driven by fast development of both virtual reality and volume visualization, we discuss some critical techniques towards building a volumetric VRsystem, specifically the modeling, rendering, and manipulations of a vol...Driven by fast development of both virtual reality and volume visualization, we discuss some critical techniques towards building a volumetric VRsystem, specifically the modeling, rendering, and manipulations of a volumetric scene.Techniques such as voxel-based object simplification, accelerated volume rendering,fast stereo volume rendering, and volumetric 'collision detection' are introduced andimproved, with the idea of demonstrating the possibilities and potential benefits ofincorporating volumetric models into VR systems.展开更多
We present VCNet,a new deep learning approach for volume completion by synthesizing missing subvolumes.Our solution leverages a generative adversarial network(GAN)that learns to complete volumes using the adversarial ...We present VCNet,a new deep learning approach for volume completion by synthesizing missing subvolumes.Our solution leverages a generative adversarial network(GAN)that learns to complete volumes using the adversarial and volumetric losses.The core design of VCNet features a dilated residual block and long-term connection.During training,VCNet first randomly masks basic subvolumes(e.g.,cuboids,slices)from complete volumes and learns to recover them.Moreover,we design a two-stage algorithm for stabilizing and accelerating network optimization.Once trained,VCNet takes an incomplete volume as input and automatically identifies and fills in the missing subvolumes with high quality.We quantitatively and qualitatively test VCNet with volumetric data sets of various characteristics to demonstrate its effectiveness.We also compare VCNet against a diffusion-based solution and two GAN-based solutions.展开更多
文摘Background A medical content-based image retrieval(CBIR)system is designed to retrieve images from large imaging repositories that are visually similar to a user′s query image.CBIR is widely used in evidence-based diagnosis,teaching,and research.Although the retrieval accuracy has largely improved,there has been limited development toward visualizing important image features that indicate the similarity of retrieved images.Despite the prevalence of 3D volumetric data in medical imaging such as computed tomography(CT),current CBIR systems still rely on 2D cross-sectional views for the visualization of retrieved images.Such 2D visualization requires users to browse through the image stacks to confirm the similarity of the retrieved images and often involves mental reconstruction of 3D information,including the size,shape,and spatial relations of multiple structures.This process is time-consuming and reliant on users'experience.Methods In this study,we proposed an importance-aware 3D volume visualization method.The rendering parameters were automatically optimized to maximize the visibility of important structures that were detected and prioritized in the retrieval process.We then integrated the proposed visualization into a CBIR system,thereby complementing the 2D cross-sectional views for relevance feedback and further analyses.Results Our preliminary results demonstrate that 3D visualization can provide additional information using multimodal positron emission tomography and computed tomography(PETCT)images of a non-small cell lung cancer dataset.
基金the Advanced Project Foundation between China and France(PRA SI03-02).
文摘In order to perform a high-quality interactive rendering of large medical data sets on a single off-the-shelf PC, a LOD selection algorithm for multi-resolution volume rendering using 3D texture mapping is presented, which uses an adaptive scheme that renders the volume in a region-of-interest at a high resolution and the volume away from this region at lower resolutions. The algorithm is based on several important criteria, and rendering is done adaptively by selecting high-resolution cells close to a center of attention and low-resolution cells away from this area. In addition, our hierarchical level-of-detail representation guarantees consistent interpolation between different resolution levels. Experiments have been applied to a number of large medical data and have produced high quality images at interactive frame rates using standard PC hardware.
文摘Direct isosurface volume rendering is the most prominent modern method for medical data visualization.It is based on finding intersection points between the rays corresponding to pixels on the screen and isosurface. This article describes a two-pass algorithm for accelerating the method on the graphic processing unit(GPU). On the first pass, the intersections with the isosurface are found only for a small number of rays, which is done by rendering into a lower-resolution texture. On the second pass, the obtained information is used to efficiently calculate the intersection points of all the other. The number of rays to use during the first pass is determined by using an adaptive algorithm, which runs on the central processing unit(CPU) in parallel with the second pass of the rendering. The proposed approach allows to significantly speed up isosurface visualization without quality loss. Experiments show acceleration up to 10 times in comparison with a common ray casting method implemented on GPU. To the authors’ knowledge, this is the fastest approach for ray casting which does not require any preprocessing and could be run on common GPUs.
基金the Program of Advance Research between France and Chinese(No.PRA SI 03-03)the Region Rhone-Alpes of France within the Project"MIRA Research 2003"the Project of Image Guided Surgery of Shanghai,China(No.045115001)
文摘Elliptical splats are used to represent and render the isosurface of volume data. The method consists of two steps. The first step is to extract points on the isosurface by looking up the case table. In the second step, properties of splats are computed based on local geometry. Rendering is achieved using surface splatting algorithm. The obtained results show that the extraction time of isosurfaces can be reduced by a factor of three. So our approach is more appropriate for interactive visualization of large medical data than the classical marching cubes (MC) technique.
基金supported in part by Hong Kong RGC CERG under Grant No. 618705
文摘Direct volume rendering (DVR) is a powerful visualization technique which allows users to effectively explore and study volumetric datasets. Different transparency settings can be flexibly assigned to different structures such that some valuable information can be revealed in direct volume rendered images (DVRIs). However, end-users often feel that some risks are always associated with DVR because they do not know whether any important information is missing from the transparent regions of DVRIs. In this paper, we investigate how to semi-automatically generate a set of DVRIs and also an animation which can reveal information missed in the original DVRIs and meanwhile satisfy some image quality criteria such as coherence. A complete framework is developed to tackle various problems related to the generation and quality evaluation of visibility-aware DVRIs and animations. Our technique can reduce the risk of using direct volume rendering and thus boost the confidence of users in volume rendering systems.
文摘Driven by fast development of both virtual reality and volume visualization, we discuss some critical techniques towards building a volumetric VRsystem, specifically the modeling, rendering, and manipulations of a volumetric scene.Techniques such as voxel-based object simplification, accelerated volume rendering,fast stereo volume rendering, and volumetric 'collision detection' are introduced andimproved, with the idea of demonstrating the possibilities and potential benefits ofincorporating volumetric models into VR systems.
基金This work was supported in part by the U.S.National Science Foundation through grants IIS-1455886,CNS-1629914,DUE-1833129,IIS-1955395,IIS-2101696,and OAC-2104158.The authors would like to thank the anonymous reviewers for their insightful comments.
文摘We present VCNet,a new deep learning approach for volume completion by synthesizing missing subvolumes.Our solution leverages a generative adversarial network(GAN)that learns to complete volumes using the adversarial and volumetric losses.The core design of VCNet features a dilated residual block and long-term connection.During training,VCNet first randomly masks basic subvolumes(e.g.,cuboids,slices)from complete volumes and learns to recover them.Moreover,we design a two-stage algorithm for stabilizing and accelerating network optimization.Once trained,VCNet takes an incomplete volume as input and automatically identifies and fills in the missing subvolumes with high quality.We quantitatively and qualitatively test VCNet with volumetric data sets of various characteristics to demonstrate its effectiveness.We also compare VCNet against a diffusion-based solution and two GAN-based solutions.