3D Reconstruction for Motion Blurred Images Using Deep Learning-Based Intelligent Systems

The 3D reconstruction using deep learning-based intelligent systems can provide great help for measuring an individual’s height and shape quickly and accurately through 2D motion-blurred images.Generally,during the acquisition of images in real-time,motion blur,caused by camera shaking or human motion,appears.Deep learning-based intelligent control applied in vision can help us solve the problem.To this end,we propose a 3D reconstruction method for motion-blurred images using deep learning.First,we develop a BF-WGAN algorithm that combines the bilateral filtering(BF)denoising theory with a Wasserstein generative adversarial network(WGAN)to remove motion blur.The bilateral filter denoising algorithm is used to remove the noise and to retain the details of the blurred image.Then,the blurred image and the corresponding sharp image are input into the WGAN.This algorithm distinguishes the motion-blurred image from the corresponding sharp image according to the WGAN loss and perceptual loss functions.Next,we use the deblurred images generated by the BFWGAN algorithm for 3D reconstruction.We propose a threshold optimization random sample consensus(TO-RANSAC)algorithm that can remove the wrong relationship between two views in the 3D reconstructed model relatively accurately.Compared with the traditional RANSAC algorithm,the TO-RANSAC algorithm can adjust the threshold adaptively,which improves the accuracy of the 3D reconstruction results.The experimental results show that our BF-WGAN algorithm has a better deblurring effect and higher efficiency than do other representative algorithms.In addition,the TO-RANSAC algorithm yields a calculation accuracy considerably higher than that of the traditional RANSAC algorithm.

Deep-Learning-Empowered 3D Reconstruction for Dehazed Images in IoT-Enhanced Smart Cities

With increasingly more smart cameras deployed in infrastructure and commercial buildings,3D reconstruction can quickly obtain cities’information and improve the efficiency of government services.Images collected in outdoor hazy environments are prone to color distortion and low contrast;thus,the desired visual effect cannot be achieved and the difficulty of target detection is increased.Artificial intelligence(AI)solutions provide great help for dehazy images,which can automatically identify patterns or monitor the environment.Therefore,we propose a 3D reconstruction method of dehazed images for smart cities based on deep learning.First,we propose a fine transmission image deep convolutional regression network(FT-DCRN)dehazing algorithm that uses fine transmission image and atmospheric light value to compute dehazed image.The DCRN is used to obtain the coarse transmission image,which can not only expand the receptive field of the network but also retain the features to maintain the nonlinearity of the overall network.The fine transmission image is obtained by refining the coarse transmission image using a guided filter.The atmospheric light value is estimated according to the position and brightness of the pixels in the original hazy image.Second,we use the dehazed images generated by the FT-DCRN dehazing algorithm for 3D reconstruction.An advanced relaxed iterative fine matching based on the structure from motion(ARI-SFM)algorithm is proposed.The ARISFM algorithm,which obtains the fine matching corner pairs and reduces the number of iterations,establishes an accurate one-to-one matching corner relationship.The experimental results show that our FT-DCRN dehazing algorithm improves the accuracy compared to other representative algorithms.In addition,the ARI-SFM algorithm guarantees the precision and improves the efficiency.

Three-Dimensional Measurement Using Structured Light Based on Deep Learning

Three-dimensional(3D)reconstruction using structured light projection has the characteristics of non-contact,high precision,easy operation,and strong real-time performance.However,for actual measurement,projection modulated images are disturbed by electronic noise or other interference,which reduces the precision of the measurement system.To solve this problem,a 3D measurement algorithm of structured light based on deep learning is proposed.The end-to-end multi-convolution neural network model is designed to separately extract the coarse-and fine-layer features of a 3D image.The point-cloud model is obtained by nonlinear regression.The weighting coefficient loss function is introduced to the multi-convolution neural network,and the point-cloud data are continuously optimized to obtain the 3D reconstruction model.To verify the effectiveness of the method,image datasets of different 3D gypsum models were collected,trained,and tested using the above method.Experimental results show that the algorithm effectively eliminates external light environmental interference,avoids the influence of object shape,and achieves higher stability and precision.The proposed method is proved to be effective for regular objects.

Deep learning for image reconstruction in thermoacoustic tomography

Microwave-induced thermoacoustic tomography(TAT)is a rapidly-developing noninvasive imaging technique that integrates the advantages of microwave imaging and ultrasound imaging.While an image reconstruction algorithm is critical for the TAT,current reconstruction methods often creates significant artifacts and are computationally costly.In this work,we propose a deep learning-based end-to-end image reconstruction method to achieve the direct reconstruction from the sinogram data to the initial pressure density image.We design a new network architecture TAT-Net to transfer the sinogram domain to the image domain with high accuracy.For the scenarios where realistic training data are scarce or unavailable,we use the finite element method(FEM)to generate synthetic data where the domain gap between the synthetic and realistic data is resolved through the signal processing method.The TAT-Net trained with synthetic data is evaluated through both simulations and phantom experiments and achieves competitive performance in artifact removal and robustness.Compared with other state-of-the-art reconstruction methods,the TAT-Net method can reduce the root mean square error to 0.0143,and increase the structure similarity and peak signal-to-noise ratio to 0.988 and 38.64,respectively.The results obtained indicate that the TAT-Net has great potential applications in improving image reconstruction quality and fast quantitative reconstruction.

Triplet Label Based Image Retrieval Using Deep Learning in Large Database

Recent days,Image retrieval has become a tedious process as the image database has grown very larger.The introduction of Machine Learning(ML)and Deep Learning(DL)made this process more comfortable.In these,the pair-wise label similarity is used tofind the matching images from the database.But this method lacks of limited propose code and weak execution of misclassified images.In order to get-rid of the above problem,a novel triplet based label that incorporates context-spatial similarity measure is proposed.A Point Attention Based Triplet Network(PABTN)is introduced to study propose code that gives maximum discriminative ability.To improve the performance of ranking,a corre-lating resolutions for the classification,triplet labels based onfindings,a spatial-attention mechanism and Region Of Interest(ROI)and small trial information loss containing a new triplet cross-entropy loss are used.From the experimental results,it is shown that the proposed technique exhibits better results in terms of mean Reciprocal Rank(mRR)and mean Average Precision(mAP)in the CIFAR-10 and NUS-WIPE datasets.

Aspect based sentiment analysis using multi-criteria decision-making and deep learning under COVID-19 pandemic in India

The COVID-19 pandemic has a significant impact on the global economy and health.While the pandemic continues to cause casualties in millions,many countries have gone under lockdown.During this period,people have to stay within walls and become more addicted towards social networks.They express their emotions and sympathy via these online platforms.Thus,popular social media(Twitter and Facebook)have become rich sources of information for Opinion Mining and Sentiment Analysis on COVID-19-related issues.We have used Aspect Based Sentiment Analysis to anticipate the polarity of public opinion underlying different aspects from Twitter during lockdown and stepwise unlock phases.The goal of this study is to find the feelings of Indians about the lockdown initiative taken by the Government of India to stop the spread of Coronavirus.India-specific COVID-19 tweets have been annotated,for analysing the sentiment of common public.To classify the Twitter data set a deep learning model has been proposed which has achieved accuracies of 82.35%for Lockdown and 83.33%for Unlock data set.The suggested method outperforms many of the contemporary approaches(long shortterm memory,Bi-directional long short-term memory,Gated Recurrent Unit etc.).This study highlights the public sentiment on lockdown and stepwise unlocks,imposed by the Indian Government on various aspects during the Corona outburst.

An Efficient Deep Learning-based Content-based Image Retrieval Framework

The use of massive image databases has increased drastically over the few years due to evolution of multimedia technology.Image retrieval has become one of the vital tools in image processing applications.Content-Based Image Retrieval(CBIR)has been widely used in varied applications.But,the results produced by the usage of a single image feature are not satisfactory.So,multiple image features are used very often for attaining better results.But,fast and effective searching for relevant images from a database becomes a challenging task.In the previous existing system,the CBIR has used the combined feature extraction technique using color auto-correlogram,Rotation-Invariant Uniform Local Binary Patterns(RULBP)and local energy.However,the existing system does not provide significant results in terms of recall and precision.Also,the computational complexity is higher for the existing CBIR systems.In order to handle the above mentioned issues,the Gray Level Co-occurrence Matrix(GLCM)with Deep Learning based Enhanced Convolution Neural Network(DLECNN)is proposed in this work.The proposed system framework includes noise reduction using histogram equalization,feature extraction using GLCM,similarity matching computation using Hierarchal and Fuzzy c-Means(HFCM)algorithm and the image retrieval using DLECNN algorithm.The histogram equalization has been used for computing the image enhancement.This enhanced image has a uniform histogram.Then,the GLCM method has been used to extract the features such as shape,texture,colour,annotations and keywords.The HFCM similarity measure is used for computing the query image vector's similarity index with every database images.For enhancing the performance of this image retrieval approach,the DLECNN algorithm is proposed to retrieve more accurate features of the image.The proposed GLCM+DLECNN algorithm provides better results associated with high accuracy,precision,recall,f-measure and lesser complexity.From the experimental results,it is clearly observed that the proposed system provides efficient image retrieval for the given query image.

Feature-Based Aggregation and Deep Reinforcement Learning:A Survey and Some New Implementations

In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features. We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration. In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations. We argue that the cost function of a policy may be approximated much more accurately by the nonlinear function of the features provided by aggregation, than by the linear function of the features provided by neural networkbased reinforcement learning, thereby potentially leading to more effective policy improvement.

Deep learning tomographic reconstruction through hierarchical decomposition of domain transforms

Deep learning(DL)has shown unprecedented performance for many image analysis and image enhancement tasks.Yet,solving large-scale inverse problems like tomographic reconstruction remains challenging for DL.These problems involve non-local and space-variant integral transforms between the input and output domains,for which no efficient neural network models are readily available.A prior attempt to solve tomographic reconstruction problems with supervised learning relied on a brute-force fully connected network and only allowed reconstruction with a 128^(4)system matrix size.This cannot practically scale to realistic data sizes such as 512^(4)and 512^(6)for three-dimensional datasets.Here we present a novel framework to solve such problems with DL by casting the original problem as a continuum of intermediate representations between the input and output domains.The original problem is broken down into a sequence of simpler transformations that can be well mapped onto an efficient hierarchical network architecture,with exponentially fewer parameters than a fully connected network would need.We applied the approach to computed tomography(CT)image reconstruction for a 5124 system matrix size.This work introduces a new kind of data-driven DL solver for full-size CT reconstruction without relying on the structure of direct(analytical)or iterative(numerical)inversion techniques.This work presents a feasibility demonstration of full-scale learnt reconstruction,whereas more developments will be needed to demonstrate superiority relative to traditional reconstruction approaches.The proposed approach is also extendable to other imaging problems such as emission and magnetic resonance reconstruction.More broadly,hierarchical DL opens the door to a new class of solvers for general inverse problems,which could potentially lead to improved signal-to-noise ratio,spatial resolution and computational efficiency in various areas.

Improved Lightweight Deep Learning Algorithm in 3D Reconstruction

The three-dimensional(3D)reconstruction technology based on structured light has been widely used in the field of industrial measurement due to its many advantages.Aiming at the problems of high mismatch rate and poor real-time performance caused by factors such as system jitter and noise,a lightweight stripe image feature extraction algorithm based on You Only Look Once v4(YOLOv4)network is proposed.First,Mobilenetv3 is used as the backbone network to effectively extract features,and then the Mish activation function and Complete Intersection over Union(CIoU)loss function are used to calculate the improved target frame regression loss,which effectively improves the accuracy and real-time performance of feature detection.Simulation experiment results show that the model size after the improved algorithm is only 52 MB,the mean average accuracy(mAP)of fringe image data reconstruction reaches 82.11%,and the 3D point cloud restoration rate reaches 90.1%.Compared with the existing model,it has obvious advantages and can satisfy the accuracy and real-time requirements of reconstruction tasks in resource-constrained equipment.

Deep-learning-based methods for super-resolution fluorescence microscopy

The algorithm used for reconstruction or resolution enhancement is one of the factors affectingthe quality of super-resolution images obtained by fluorescence microscopy.Deep-learning-basedalgorithms have achieved stateof-the-art performance in super-resolution fluorescence micros-copy and are becoming increasingly attractive.We firstly introduce commonly-used deep learningmodels,and then review the latest applications in terms of the net work architectures,the trainingdata and the loss functions.Additionally,we discuss the challenges and limits when using deeplearning to analyze the fluorescence microscopic data,and suggest ways to improve the reliability and robustness of deep learning applications.

Deep Learning Framework for Classification of Emoji Based Sentiments

Recent patterns of human sentiments are highly influenced by emoji based sentiments(EBS).Social media users are widely using emoji based sentiments(EBS)in between text messages,tweets and posts.Although tiny pictures of emoji contains sufficient information to be considered for construction of classification model;but due to the wide range of dissimilar,heterogynous and complex patterns of emoji with similarmeanings(SM)have become one of the significant research areas of machine vision.This paper proposes an approach to provide meticulous assistance to social media application(SMA)users to classify the EBS sentiments.Proposed methodology consists upon three layerswhere first layer deals with data cleaning and feature selection techniques to detect dissimilar emoji patterns(DEP)with similar meanings(SM).In first sub step we input set of emoji,in second sub step every emoji has to qualify user defined threshold,in third sub step algorithm detects every emoji by considering as objects and in fourth step emoji images are cropped,after data cleaning these tiny images are saved as emoji images.In second step we build classification model by using convolutional neural networks(CNN)to explore hidden knowledge of emoji datasets.In third step we present results visualization by using confusion matrix and other estimations.This paper contributes(1)data cleaning method to detect EBS;(2)highest classification accuracy for emoji classification measured as 97.63%.

Current status of deep learning in abdominal image reconstruction

Abdominal magnetic resonance imaging(MRI)and computed tomography(CT)are commonly used for disease screening,diagnosis,and treatment guidance.However,abdominal MRI has disadvantages including slow speed and vulnerability to motions,while CT suffers from problems of radiation.It has been reported that deep learning reconstruction can solve such problems while maintaining good image quality.Recently,deep learning-based image reconstruction has become a hot topic in the field of medical imaging.This study reviews the latest research on deep learning reconstruction in abdominal imaging,including the widely used convolutional neural network,generative adversarial network,and recurrent neural network.

Advanced 4-dimensional cone-beam computed tomography reconstruction by combining motion estimation, motioncompensated reconstruction, biomechanical modeling and deep learning

4-Dimensional cone-beam computed tomography(4D-CBCT)offers several key advantages over conventional 3DCBCT in moving target localization/delineation,structure de-blurring,target motion tracking,treatment dose accumulation and adaptive radiation therapy.However,the use of the 4D-CBCT in current radiation therapy practices has been limited,mostly due to its sub-optimal image quality from limited angular sampling of conebeam projections.In this study,we summarized the recent developments of 4D-CBCT reconstruction techniques for image quality improvement,and introduced our developments of a new 4D-CBCT reconstruction technique which features simultaneous motion estimation and image reconstruction(SMEIR).Based on the original SMEIR scheme,biomechanical modeling-guided SMEIR(SMEIR-Bio)was introduced to further improve the reconstruction accuracy of fine details in lung 4D-CBCTs.To improve the efficiency of reconstruction,we recently developed a U-net-based deformation-vector-field(DVF)optimization technique to leverage a population-based deep learning scheme to improve the accuracy of intra-lung DVFs(SMEIR-Unet),without explicit biomechanical modeling.Details of each of the SMEIR,SMEIR-Bio and SMEIR-Unet techniques were included in this study,along with the corresponding results comparing the reconstruction accuracy in terms of CBCT images and the DVFs.We also discussed the application prospects of the SMEIR-type techniques in image-guided radiation therapy and adaptive radiation therapy,and presented potential schemes on future developments to achieve faster and more accurate 4D-CBCT imaging.

Toward Improved Accuracy in Quasi-Static Elastography Using Deep Learning

Elastography is a non-invasive medical imaging technique to map the spatial variation of elastic properties of soft tissues.The quality of reconstruction results in elastography is highly sensitive to the noise induced by imaging measurements and processing.To address this issue,we propose a deep learning(DL)model based on conditional Generative Adversarial Networks(cGANs)to improve the quality of nonhomogeneous shear modulus reconstruction.To train this model,we generated a synthetic displacement field with finite element simulation under known nonhomogeneous shear modulus distribution.Both the simulated and experimental displacement fields are used to validate the proposed method.The reconstructed results demonstrate that the DL model with synthetic training data is able to improve the quality of the reconstruction compared with the well-established optimization method.Moreover,we emphasize that our DL model is only trained on synthetic data.This might provide a way to alleviate the challenge of obtaining clinical or experimental data in elastography.Overall,this work addresses several fatal issues in applying the DL technique into elastography,and the proposed method has shown great potential in improving the accuracy of the disease diagnosis in clinical medicine.

Fast Mesh Reconstruction from Single View Based on GCN and Topology Modification

3D reconstruction based on single view aims to reconstruct the entire 3D shape of an object from one perspective.When existing methods reconstruct the mesh surface of complex objects,the surface details are difficult to predict and the reconstruction visual effect is poor because the mesh representation is not easily integrated into the deep learning framework;the 3D topology is easily limited by predefined templates and inflexible,and unnecessary mesh self-intersections and connections will be generated when reconstructing complex topology,thus destroying the surface details;the training of the reconstruction network is limited by the large amount of information attached to the mesh vertices,and the training time of the reconstructed network is too long.In this paper,we propose a method for fast mesh reconstruction from single view based on Graph Convolutional Network(GCN)and topology modification.We use GCN to ensure the generation of high-quality mesh surfaces and use topology modification to improve the flexibility of the topology.Meanwhile,a feature fusion method is proposed to make full use of the features of each stage of the image hierarchically.We use 3D open dataset ShapeNet to train our network and add a new weight parameter to speed up the training process.Extensive experiments demonstrate that our method can not only reconstruct object meshes on complex topological surfaces,but also has better qualitative and quantitative results.

Sparse Seismic Data Reconstruction Based on a Convolutional Neural Network Algorithm

At present,the acquisition of seismic data is developing toward high-precision and high-density methods.However,complex natural environments and cultural factors in many exploration areas cause difficulties in achieving uniform and intensive acquisition,which makes complete seismic data collection impossible.Therefore,data reconstruction is required in the processing link to ensure imaging accuracy.Deep learning,as a new field in rapid development,presents clear advantages in feature extraction and modeling.In this study,the convolutional neural network deep learning algorithm is applied to seismic data reconstruction.Based on the convolutional neural network algorithm and combined with the characteristics of seismic data acquisition,two training strategies of supervised and unsupervised learning are designed to reconstruct sparse acquisition seismic records.First,a supervised learning strategy is proposed for labeled data,wherein the complete seismic data are segmented as the input of the training set and are randomly sampled before each training,thereby increasing the number of samples and the richness of features.Second,an unsupervised learning strategy based on large samples is proposed for unlabeled data,and the rolling segmentation method is used to update(pseudo)labels and training parameters in the training process.Through the reconstruction test of simulated and actual data,the deep learning algorithm based on a convolutional neural network shows better reconstruction quality and higher accuracy than compressed sensing based on Curvelet transform.

Deep Learned Singular Residual Network for Super Resolution Reconstruction

Single image super resolution(SISR)techniques produce images of high resolution(HR)as output from input images of low resolution(LR).Motivated by the effectiveness of deep learning methods,we provide a framework based on deep learning to achieve super resolution(SR)by utilizing deep singular-residual neural network(DSRNN)in training phase.Residuals are obtained from the difference between HR and LR images to generate LR-residual example pairs.Singular value decomposition(SVD)is applied to each LR-residual image pair to decompose into subbands of low and high frequency components.Later,DSRNN is trained on these subbands through input and output channels by optimizing the weights and biases of the network.With fewer layers in DSRNN,the influence of exploding gradients is reduced.This speeds up the learning process and also improves accuracy by using skip connections.The trained DSRNN parameters yield residuals to recover the HR subbands in the testing phase.Experimental analysis shows that the proposed method results in superior performance to existingmethods in terms of subjective quality.Extensive testing results on popular benchmark datasets such as set5,set14,and urban100 for a scaling factor of 4 show the effectiveness of the proposed method across different qualitative evaluation metrics.

Triple-path feature transform network for ring-array photoacoustic tomography image reconstruction

Photoacoustic imaging(PAI)is a noninvasive emerging imaging method based on the photoacoustic effect,which provides necessary assistance for medical diagnosis.It has the characteristics of large imaging depth and high contrast.However,limited by the equipment cost and reconstruction time requirements,the existing PAI systems distributed with annular array transducers are difficult to take into account both the image quality and the imaging speed.In this paper,a triple-path feature transform network(TFT-Net)for ring-array photoacoustic tomography is proposed to enhance the imaging quality from limited-view and sparse measurement data.Specifically,the network combines the raw photoacoustic pressure signals and conventional linear reconstruction images as input data,and takes the photoacoustic physical model as a prior information to guide the reconstruction process.In addition,to enhance the ability of extracting signal features,the residual block and squeeze and excitation block are introduced into the TFT-Net.For further efficient reconstruction,the final output of photoacoustic signals uses‘filter-then-upsample’operation with a pixel-shuffle multiplexer and a max out module.Experiment results on simulated and in-vivo data demonstrate that the constructed TFT-Net can restore the target boundary clearly,reduce background noise,and realize fast and high-quality photoacoustic image reconstruction of limited view with sparse sampling.

Advances in deep learning for computed tomography denoising

Computed tomography(CT)has seen a rapid increase in use in recent years.Radiation from CT accounts for a significant proportion of total medical radiation.However,given the known harmful impact of radiation exposure to the human body,the excessive use of CT in medical environments raises concerns.Concerns over increasing CT use and its associated radiation burden have prompted efforts to reduce radiation dose during the procedure.Therefore,low-dose CT has attracted major attention in the radiology,since CT-associated x-ray radiation carries health risks for patients.The reduction of the CT radiation dose,however,compromises the signal-to-noise ratio,which affects image quality and diagnostic performance.Therefore,several denoising methods have been developed and applied to image processing technologies with the goal of reducing image noise.Recently,deep learning applications that improve image quality by reducing the noise and artifacts have become commercially available for diagnostic imaging.Deep learning image reconstruction shows great potential as an advanced reconstruction method to improve the quality of clinical CT images.These improvements can provide significant benefit to patients regardless of their disease,and further advances are expected in the near future.