Special issue “Photoacoustic imaging: microscopy, tomography, and their recent applications in biomedicine” in visual computation for industry, biomedicine, and art

Photoacoustic(PA)imaging is a promising non-invasive and non-ionizing biomedical imaging modality that emerged in recent years.The articles presented in this special issue describe some of newest progress in this field.We are extremely grateful to all contributing authors.The first part of the issue covers new laser source devel-opment,including fiber lasers and laser diodes.The sec-ond part is dedicated to improving the image resolution through chronic cranial window techniques,virtual-point concept,fast polygon scanning,and Fabry Perot sensing.The third part shows the basic principles of photoacous-tic/ultrasound imaging and its applications.

Two fully automated data-driven 3D whole-breast segmentation strategies in MRI for MR-based breast density using image registration and U-Net with a focus on reproducibility

Presence of higher breast density(BD)and persistence over time are risk factors for breast cancer.A quantitatively accurate and highly reproducible BD measure that relies on precise and reproducible whole-breast segmentation is desirable.In this study,we aimed to develop a highly reproducible and accurate whole-breast segmentation algorithm for the generation of reproducible BD measures.Three datasets of volunteers from two clinical trials were included.Breast MR images were acquired on 3T Siemens Biograph mMR,Prisma,and Skyra using 3D Cartesian six-echo GRE sequences with a fat-water separation technique.Two whole-breast segmentation strategies,utiliz-ing image registration and 3D U-Net,were developed.Manual segmentation was performed.A task-based analysis was performed:a previously developed MR-based BD measure,MagDensity,was calculated and assessed using automated and manual segmentation.The mean squared error(MSE)and intraclass correlation coefficient(ICC)between MagDensity were evaluated using the manual segmentation as a reference.The test-retest reproducibility of MagDensity derived from different breast segmentation methods was assessed using the difference between the test and retest measures(Δ_(2-1)),MSE,and ICC.The results showed that MagDensity derived by the registration and deep learning segmentation methods exhibited high concordance with manual segmentation,with ICCs of 0.986(95%CI:0.974-0.993)and 0.983(95%CI:0.961-0.992),respectively.For test-retest analysis,MagDensity derived using the regis-tration algorithm achieved the smallest MSE of 0.370 and highest ICC of 0.993(95%CI:0.982-0.997)when compared to other segmentation methods.In conclusion,the proposed registration and deep learning whole-breast segmentation methods are accurate and reliable for estimating BD.Both methods outperformed a previously developed algorithm and manual segmentation in the test-retest assessment,with the registration exhibiting superior performance for highly reproducible BD measurements.

A survey: which features are required for dynamic visual simultaneous localization and mapping?

In recent years,simultaneous localization and mapping in dynamic environments(dynamic SLAM)has attracted significant attention from both academia and industry.Some pioneering work on this technique has expanded the potential of robotic applications.Compared to standard SLAM under the static world assumption,dynamic SLAM divides features into static and dynamic categories and leverages each type of feature properly.Therefore,dynamic SLAM can provide more robust localization for intelligent robots that operate in complex dynamic environments.Additionally,to meet the demands of some high-level tasks,dynamic SLAM can be integrated with multiple object tracking.This article presents a survey on dynamic SLAM from the perspective of feature choices.A discussion of the advantages and disadvantages of different visual features is provided in this article.

Influence of postural changes on haemodynamics in internal carotid artery bifurcation aneurysm using numerical methods

Cerebral intracranial aneurysms are serious problems that can lead to stroke,coma,and even death.The effect of blood flow on cerebral aneurysms and their relationship with rupture are unknown.In addition,postural changes and their relevance to haemodynamics of blood flow are difficult to measure in vivo using clinical imaging alone.Computational simulations investigating the detailed haemodynamics in cerebral aneurysms have been developed in recent times not only to understand the progression and rupture but also for clinical evaluation and treatment.In the present study,the haemodynamics of a patient-specific case of a large aneurysm on the left side internal carotid bifurcation(LICA)and no aneurysm on the right side internal carotid bifurcation(RICA)was investigated.The simulation of these patient-specific models using fluid–structure interaction provides a valuable comparison of flow behavior between normal and aneurysm models.The influences of postural changes were investigated during standing,sleeping,and head-down(HD)position.Significant changes in flow were observed during the HD position and quit high arterial blood pressure in the internal carotid artery(ICA)aneurysm model was established when compared to the normal ICA model.The velocity increased abruptly during the HD position by more than four times(LICA and RICA)and wall shear stress by four times(LICA)to ten times(RICA).The complex spiral flow and higher pressures prevailing within the dome increase the risk of aneurysm rupture.

A projection-domain iterative algorithm for metal artifact reduction by minimizing the total-variation norm and the negative-pixel energy

Metal objects in X-ray computed tomography can cause severe artifacts.The state-of-the-art metal artifact reduction methods are in the sinogram inpainting category and are iterative methods.This paper proposes a projectiondomain algorithm to reduce the metal artifacts.In this algorithm,the unknowns are the metal-affected projections,while the objective function is set up in the image domain.The data fidelity term is not utilized in the objective function.The objective function of the proposed algorithm consists of two terms:the total variation of the metalremoved image and the energy of the negative-valued pixels in the image.After the metal-affected projections are modified,the final image is reconstructed via the filtered backprojection algorithm.The feasibility of the proposed algorithm has been verified by real experimental data.

A comprehensive review of machine learning techniques on diabetes detection

Diabetes mellitus has been an increasing concern owing to its high morbidity,and the average age of individual affected by of individual affected by this disease has now decreased to mid-twenties.Given the high prevalence,it is necessary to address with this problem effectively.Many researchers and doctors have now developed detection techniques based on artificial intelligence to better approach problems that are missed due to human errors.Data mining techniques with algorithms such as-density-based spatial clustering of applications with noise and ordering points to identify the cluster structure,the use of machine vision systems to learn data on facial images,gain better features for model training,and diagnosis via presentation of iridocyclitis for detection of the disease through iris patterns have been deployed by various practitioners.Machine learning classifiers such as support vector machines,logistic regression,and decision trees,have been comparative discussed various authors.Deep learning models such as artificial neural networks and recurrent neural networks have been considered,with primary focus on long short-term memory and convolutional neural network architectures in comparison with other machine learning models.Various parameters such as the root-mean-square error,mean absolute errors,area under curves,and graphs with varying criteria are commonly used.In this study,challenges pertaining to data inadequacy and model deployment are discussed.The future scope of such methods has also been discussed,and new methods are expected to enhance the performance of existing models,allowing them to attain greater insight into the conditions on which the prevalence of the disease depends.

Spatial weight matrix in dimensionality reduction reconstruction for microelectromechanical system-based photoacoustic microscopy

A micro-electromechanical system(MEMS)scanning mirror accelerates the raster scanning of optical-resolution photoacoustic microscopy(OR-PAM).However,the nonlinear tilt angular-voltage characteristic of a MEMS mirror introduces distortion into the maximum back-projection image.Moreover,the size of the airy disk,ultrasonic sensor properties,and thermal effects decrease the resolution.Thus,in this study,we proposed a spatial weight matrix(SWM)with a dimensionality reduction for image reconstruction.The three-layer SWM contains the invariable information of the system,which includes a spatial dependent distortion correction and 3D deconvolution.We employed an ordinal-valued Markov random field and the Harris Stephen algorithm,as well as a modified delay-and-sum method during a time reversal.The results from the experiments and a quantitative analysis demonstrate that images can be effectively reconstructed using an SWM;this is also true for severely distorted images.The index of the mutual information between the reference images and registered images was 70.33 times higher than the initial index,on average.Moreover,the peak signal-to-noise ratio was increased by 17.08%after 3D deconvolution.This accomplishment offers a practical approach to image reconstruction and a promising method to achieve a real-time distortion correction for MEMS-based OR-PAM.

A non-uniform allowance allocation method based on interim state stiffness of machining features for NC programming of structural parts

For thin-walled parts,uniform allowance to each machining surface is allocated by the traditional machining method.Considering the sequence of the adjacent machining features,it may cause poor stiffness for some side walls due to a minor wall thickness,which may cause the deformation of the final formed parts to be large,or deduce machining efficiency for some machining features due to too thick remains.In order to address this issue,a non-uniform allowance allocation method based on interim state stiffness of machining features for the finishing of thin-walled structural parts is proposed in this paper.In this method,the interim state model of machining features is constructed according to the machining sequence of the parts,and the stiffness of the side wall is taken as the evaluation index to allocate reasonable allowance value to the corresponding machining surface to ensure the stiffness requirement of the parts in the machining process.According to the finite element simulation results,the non-uniform allowance allocation method proposed in this paper can effectively improve the stiffness of the parts and reduce the deformation of the parts,when compared with the traditional uniform allowance machining method.

Correction to: Recent developments in photoacoustic imaging and sensing for nondestructive testing and evaluation

Following publication of the original article[1],the authors identified an error in the article title.The first word ‘Review’ is added mistakenly by the typesetter.

Brief review of image denoising techniques

With the explosion in the number of digital images taken every day,the demand for more accurate and visually pleasing images is increasing.However,the images captured by modern cameras are inevitably degraded by noise,which leads to deteriorated visual image quality.Therefore,work is required to reduce noise without losing image features(edges,corners,and other sharp structures).So far,researchers have already proposed various methods for decreasing noise.Each method has its own advantages and disadvantages.In this paper,we summarize some important research in the field of image denoising.First,we give the formulation of the image denoising problem,and then we present several image denoising techniques.In addition,we discuss the characteristics of these techniques.Finally,we provide several promising directions for future research.

An efficient data structure for calculation of unstructured T-spline surfaces

To overcome the topological constraints of non-uniform rational B-splines,T-splines have been proposed to define the freeform surfaces.The introduction of T-junctions and extraordinary points makes it possible to represent arbitrarily shaped models by a single T-spline surface.Whereas,the complexity and flexibility of topology structure bring difficulty in programming,which have caused a great obstacle for the development and application of T-spline technologies.So far,research literatures concerning T-spline data structures compatible with extraordinary points are very scarce.In this paper,an efficient data structure for calculation of unstructured T-spline surfaces is developed,by which any complex T-spline surface models can be easily and efficiently computed.Several unstructured T-spline surface models are calculated and visualized in our prototype system to verify the validity of the proposed method.

Automatic quantification of superficial foveal avascular zone in optical coherence tomography angiography implemented with deep learning

An accurate segmentation and quantification of the superficial foveal avascular zone(sFAZ)is important to facilitate the diagnosis and treatment of many retinal diseases,such as diabetic retinopathy and retinal vein occlusion.We proposed a method based on deep learning for the automatic segmentation and quantification of the sFAZ in optical coherence tomography angiography(OCTA)images with robustness to brightness and contrast(B/C)variations.A dataset of 405 OCTA images from 45 participants was acquired with Zeiss Cirrus HD-OCT 5000 and the ground truth(GT)was manually segmented subsequently.A deep learning network with an encoder–decoder architecture was created to classify each pixel into an sFAZ or non-sFAZ class.Subsequently,we applied largestconnected-region extraction and hole-filling to fine-tune the automatic segmentation results.A maximum mean dice similarity coefficient(DSC)of 0.976±0.011 was obtained when the automatic segmentation results were compared against the GT.The correlation coefficient between the area calculated from the automatic segmentation results and that calculated from the GT was 0.997.In all nine parameter groups with various brightness/contrast,all the DSCs of the proposed method were higher than 0.96.The proposed method achieved better performance in the sFAZ segmentation and quantification compared to two previously reported methods.In conclusion,we proposed and successfully verified an automatic sFAZ segmentation and quantification method based on deep learning with robustness to B/C variations.For clinical applications,this is an important progress in creating an automated segmentation and quantification applicable to clinical analysis.

Deep-learning-based motion-correction algorithm in optical resolution photoacoustic microscopy

In this study,we propose a deep-learning-based method to correct motion artifacts in optical resolution photoacoustic microscopy(OR-PAM).The method is a convolutional neural network that establishes an end-to-end map from input raw data with motion artifacts to output corrected images.First,we performed simulation studies to evaluate the feasibility and effectiveness of the proposed method.Second,we employed this method to process images of rat brain vessels with multiple motion artifacts to evaluate its performance for in vivo applications.The results demonstrate that this method works well for both large blood vessels and capillary networks.In comparison with traditional methods,the proposed method in this study can be easily modified to satisfy different scenarios of motion corrections in OR-PAM by revising the training sets.

Energy enhanced tissue texture in spectral computed tomography for lesion classification

Tissue texture reflects the spatial distribution of contrasts of image voxel gray levels,i.e.,the tissue heterogeneity,and has been recognized as important biomarkers in various clinical tasks.Spectral computed tomography(CT)is believed to be able to enrich tissue texture by providing different voxel contrast images using different X-ray energies.Therefore,this paper aims to address two related issues for clinical usage of spectral CT,especially the photon counting CT(PCCT):(1)texture enhancement by spectral CT image reconstruction,and(2)spectral energy enriched tissue texture for improved lesion classification.For issue(1),we recently proposed a tissue-specific texture prior in addition to low rank prior for the individual energy-channel low-count image reconstruction problems in PCCT under the Bayesian theory.Reconstruction results showed the proposed method outperforms existing methods of total variation(TV),low-rank TV and tensor dictionary learning in terms of not only preserving texture features but also suppressing image noise.For issue(2),this paper will investigate three models to incorporate the enriched texture by PCCT in accordance with three types of inputs:one is the spectral images,another is the cooccurrence matrices(CMs)extracted from the spectral images,and the third one is the Haralick features(HF)extracted from the CMs.Studies were performed on simulated photon counting data by introducing attenuationenergy response curve to the traditional CT images from energy integration detectors.Classification results showed the spectral CT enriched texture model can improve the area under the receiver operating characteristic curve(AUC)score by 7.3%,0.42%and 3.0%for the spectral images,CMs and HFs respectively on the five-energy spectral data over the original single energy data only.The CM-and HF-inputs can achieve the best AUC of 0.934 and 0.927.This texture themed study shows the insight that incorporating clinical important prior information,e.g.,tissue texture in this paper,into the medical imaging,such as the upstream image reconstruction,the downstream diagnosis,and so on,can benefit the clinical tasks.

Modeling and simulation of an anatomy teaching system

Specimen observation and dissection have been regarded as the best approach to teach anatomy,but due to the severe lack of anatomical specimens in recent years,the quality of anatomy teaching has been seriously affected.In order to disseminate anatomical knowledge effectively under such circumstances,this study discusses three key factors(modeling,perception,and interaction)involved in constructing virtual anatomy teaching systems in detail.To ensure the authenticity,integrity,and accuracy of modeling,detailed three-dimensional(3D)digital anatomical models are constructed using multi-scale data,such as the Chinese Visible Human dataset,clinical imaging data,tissue sections,and other sources.The anatomical knowledge ontology is built according to the needs of the particular teaching purposes.Various kinds of anatomical knowledge and 3D digital anatomical models are organically combined to construct virtual anatomy teaching system by means of virtual reality equipment and technology.The perception of knowledge is realized by the Yi Chuang Digital Human Anatomy Teaching System that we have created.The virtual interaction mode,which is similar to actual anatomical specimen observation and dissection,can enhance the transmissibility of anatomical knowledge.This virtual anatomy teaching system captures the three key factors.It can provide realistic and reusable teaching resources,expand the new medical education model,and effectively improve the quality of anatomy teaching.

Visual analytics tool for the interpretation of hidden states in recurrent neural networks

In this paper,we introduce a visual analytics approach aimed at helping machine learning experts analyze the hidden states of layers in recurrent neural networks.Our technique allows the user to interactively inspect how hidden states store and process information throughout the feeding of an input sequence into the network.The technique can help answer questions,such as which parts of the input data have a higher impact on the prediction and how the model correlates each hidden state configuration with a certain output.Our visual analytics approach comprises several components:First,our input visualization shows the input sequence and how it relates to the output(using color coding).In addition,hidden states are visualized through a nonlinear projection into a 2-D visualization space using t-distributed stochastic neighbor embedding to understand the shape of the space of the hidden states.Trajectories are also employed to show the details of the evolution of the hidden state configurations.Finally,a time-multi-class heatmap matrix visualizes the evolution of the expected predictions for multi-class classifiers,and a histogram indicates the distances between the hidden states within the original space.The different visualizations are shown simultaneously in multiple views and support brushing-and-linking to facilitate the analysis of the classifications and debugging for misclassified input sequences.To demonstrate the capability of our approach,we discuss two typical use cases for long short-term memory models applied to two widely used natural language processing datasets.

Optical neuroimaging:advancing transcranial magnetic stimulation treatments of psychiatric disorders

Transcranial magnetic stimulation(TMS)has been established as an important and effective treatment for various psychiatric disorders.However,its effectiveness has likely been limited due to the dearth of neuronavigational tools for targeting purposes,unclear ideal stimulation parameters,and a lack of knowledge regarding the physiological response of the brain to TMS in each psychiatric condition.Modern optical imaging modalities,such as functional near-infrared spectroscopy and diffuse optical tomography,are promising tools for the study of TMS optimization and functional targeting in psychiatric disorders.They possess a unique combination of high spatial and temporal resolutions,portability,real-time capability,and relatively low costs.In this mini-review,we discuss the advent of optical imaging techniques and their innovative use in several psychiatric conditions including depression,panic disorder,phobias,and eating disorders.With further investment and research in the development of these optical imaging approaches,their potential will be paramount for the advancement of TMS treatment protocols in psychiatry.

Visualizing risk factors of dementia from scholarly literature using knowledge maps and next-generation data models

Scholarly communication of knowledge is predominantly document-based in digital repositories,and researchers find it tedious to automatically capture and process the semantics among related articles.Despite the present digital era of big data,there is a lack of visual representations of the knowledge present in scholarly articles,and a time-saving approach for a literature search and visual navigation is warranted.The majority of knowledge display tools cannot cope with current big data trends and pose limitations in meeting the requirements of automatic knowledge representation,storage,and dynamic visualization.To address this limitation,the main aim of this paper is to model the visualization of unstructured data and explore the feasibility of achieving visual navigation for researchers to gain insight into the knowledge hidden in scientific articles of digital repositories.Contemporary topics of research and practice,including modifiable risk factors leading to a dramatic increase in Alzheimer’s disease and other forms of dementia,warrant deeper insight into the evidence-based knowledge available in the literature.The goal is to provide researchers with a visual-based easy traversal through a digital repository of research articles.This paper takes the first step in proposing a novel integrated model using knowledge maps and next-generation graph datastores to achieve a semantic visualization with domain-specific knowledge,such as dementia risk factors.The model facilitates a deep conceptual understanding of the literature by automatically establishing visual relationships among the extracted knowledge from the big data resources of research articles.It also serves as an automated tool for a visual navigation through the knowledge repository for faster identification of dementia risk factors reported in scholarly articles.Further,it facilitates a semantic visualization and domain-specific knowledge discovery from a large digital repository and their associations.In this study,the implementation of the proposed model in the Neo4j graph data repository,along with the results achieved,is presented as a proof of concept.Using scholarly research articles on dementia risk factors as a case study,automatic knowledge extraction,storage,intelligent search,and visual navigation are illustrated.The implementation of contextual knowledge and its relationship for a visual exploration by researchers show promising results in the knowledge discovery of dementia risk factors.Overall,this study demonstrates the significance of a semantic visualization with the effective use of knowledge maps and paves the way for extending visual modeling capabilities in the future.

Examining data visualization pitfalls in scientific publications

Data visualization blends art and science to convey stories from data via graphical representations.Considering different problems,applications,requirements,and design goals,it is challenging to combine these two components at their full force.While the art component involves creating visually appealing and easily interpreted graphics for users,the science component requires accurate representations of a large amount of input data.With a lack of the science component,visualization cannot serve its role of creating correct representations of the actual data,thus leading to wrong perception,interpretation,and decision.It might be even worse if incorrect visual representations were intentionally produced to deceive the viewers.To address common pitfalls in graphical representations,this paper focuses on identifying and understanding the root causes of misinformation in graphical representations.We reviewed the misleading data visualization examples in the scientific publications collected from indexing databases and then projected them onto the fundamental units of visual communication such as color,shape,size,and spatial orientation.Moreover,a text mining technique was applied to extract practical insights from common visualization pitfalls.Cochran’s Q test and McNemar’s test were conducted to examine if there is any difference in the proportions of common errors among color,shape,size,and spatial orientation.The findings showed that the pie chart is the most misused graphical representation,and size is the most critical issue.It was also observed that there were statistically significant differences in the proportion of errors among color,shape,size,and spatial orientation.

Acral melanoma detection using dermoscopic images and convolutional neural networks

Acral melanoma(AM)is a rare and lethal type of skin cancer.It can be diagnosed by expert dermatologists,using dermoscopic imaging.It is challenging for dermatologists to diagnose melanoma because of the very minor differences between melanoma and non-melanoma cancers.Most of the research on skin cancer diagnosis is related to the binary classification of lesions into melanoma and non-melanoma.However,to date,limited research has been conducted on the classification of melanoma subtypes.The current study investigated the effectiveness of dermoscopy and deep learning in classifying melanoma subtypes,such as,AM.In this study,we present a novel deep learning model,developed to classify skin cancer.We utilized a dermoscopic image dataset from the Yonsei University Health System South Korea for the classification of skin lesions.Various image processing and data augmentation techniques have been applied to develop a robust automated system for AM detection.Our custombuilt model is a seven-layered deep convolutional network that was trained from scratch.Additionally,transfer learning was utilized to compare the performance of our model,where AlexNet and ResNet-18 were modified,fine-tuned,and trained on the same dataset.We achieved improved results from our proposed model with an accuracy of more than 90%for AM and benign nevus,respectively.Additionally,using the transfer learning approach,we achieved an average accuracy of nearly 97%,which is comparable to that of state-of-the-art methods.From our analysis and results,we found that our model performed well and was able to effectively classify skin cancer.Our results show that the proposed system can be used by dermatologists in the clinical decision-making process for the early diagnosis of AM.