Free form deformation and symmetry constraint‐based multimodal brain image registration using generative adversarial nets

Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.

Feature-Based Augmentation in Sarcasm Detection Using Reverse Generative Adversarial Network

Sarcasm detection in text data is an increasingly vital area of research due to the prevalence of sarcastic content in online communication.This study addresses challenges associated with small datasets and class imbalances in sarcasm detection by employing comprehensive data pre-processing and Generative Adversial Network(GAN)based augmentation on diverse datasets,including iSarcasm,SemEval-18,and Ghosh.This research offers a novel pipeline for augmenting sarcasm data with Reverse Generative Adversarial Network(RGAN).The proposed RGAN method works by inverting labels between original and synthetic data during the training process.This inversion of labels provides feedback to the generator for generating high-quality data closely resembling the original distribution.Notably,the proposed RGAN model exhibits performance on par with standard GAN,showcasing its robust efficacy in augmenting text data.The exploration of various datasets highlights the nuanced impact of augmentation on model performance,with cautionary insights into maintaining a delicate balance between synthetic and original data.The methodological framework encompasses comprehensive data pre-processing and GAN-based augmentation,with a meticulous comparison against Natural Language Processing Augmentation(NLPAug)as an alternative augmentation technique.Overall,the F1-score of our proposed technique outperforms that of the synonym replacement augmentation technique using NLPAug.The increase in F1-score in experiments using RGAN ranged from 0.066%to 1.054%,and the use of standard GAN resulted in a 2.88%increase in F1-score.The proposed RGAN model outperformed the NLPAug method and demonstrated comparable performance to standard GAN,emphasizing its efficacy in text data augmentation.

Automated Video Generation of Moving Digits from Text Using Deep Deconvolutional Generative Adversarial Network

Generating realistic and synthetic video from text is a highly challenging task due to the multitude of issues involved,including digit deformation,noise interference between frames,blurred output,and the need for temporal coherence across frames.In this paper,we propose a novel approach for generating coherent videos of moving digits from textual input using a Deep Deconvolutional Generative Adversarial Network(DD-GAN).The DDGAN comprises a Deep Deconvolutional Neural Network(DDNN)as a Generator(G)and a modified Deep Convolutional Neural Network(DCNN)as a Discriminator(D)to ensure temporal coherence between adjacent frames.The proposed research involves several steps.First,the input text is fed into a Long Short Term Memory(LSTM)based text encoder and then smoothed using Conditioning Augmentation(CA)techniques to enhance the effectiveness of the Generator(G).Next,using a DDNN to generate video frames by incorporating enhanced text and random noise and modifying a DCNN to act as a Discriminator(D),effectively distinguishing between generated and real videos.This research evaluates the quality of the generated videos using standard metrics like Inception Score(IS),Fréchet Inception Distance(FID),Fréchet Inception Distance for video(FID2vid),and Generative Adversarial Metric(GAM),along with a human study based on realism,coherence,and relevance.By conducting experiments on Single-Digit Bouncing MNIST GIFs(SBMG),Two-Digit Bouncing MNIST GIFs(TBMG),and a custom dataset of essential mathematics videos with related text,this research demonstrates significant improvements in both metrics and human study results,confirming the effectiveness of DD-GAN.This research also took the exciting challenge of generating preschool math videos from text,handling complex structures,digits,and symbols,and achieving successful results.The proposed research demonstrates promising results for generating coherent videos from textual input.

A Sketch-Based Generation Model for Diverse Ceramic Tile Images Using Generative Adversarial Network

Ceramic tiles are one of the most indispensable materials for interior decoration.The ceramic patterns can’t match the design requirements in terms of diversity and interactivity due to their natural textures.In this paper,we propose a sketch-based generation method for generating diverse ceramic tile images based on a hand-drawn sketches using Generative Adversarial Network(GAN).The generated tile images can be tailored to meet the specific needs of the user for the tile textures.The proposed method consists of four steps.Firstly,a dataset of ceramic tile images with diverse distributions is created and then pre-trained based on GAN.Secondly,for each ceramic tile image in the dataset,the corresponding sketch image is generated and then the mapping relationship between the images is trained based on a sketch extraction network using ResNet Block and jump connection to improve the quality of the generated sketches.Thirdly,the sketch style is redefined according to the characteristics of the ceramic tile images and then double cross-domain adversarial loss functions are employed to guide the ceramic tile generation network for fitting in the direction of the sketch style and to improve the training speed.Finally,we apply hidden space perturbation and interpolation for further enriching the output textures style and satisfying the concept of“one style with multiple faces”.We conduct the training process of the proposed generation network on 2583 ceramic tile images dataset.To measure the generative diversity and quality,we use Frechet Inception Distance(FID)and Blind/Referenceless Image Spatial Quality Evaluator(BRISQUE)metrics.The experimental results prove that the proposed model greatly enhances the generation results of the ceramic tile images,with FID of 32.47 and BRISQUE of 28.44.

A Generative Adversarial Nets Method for Monitoring Data Generation on Aircraft Engines

A sufficient sample size of monitoring data becomes a key factor for describing aircraft engines state.Generative adversarial nets(GAN)can be used to expand the sample size based on the existing state monitoring information.In the paper,a GAN model is introduced to design an algorithm for generating the monitoring data of aircraft engines.This feasibility of the method is illustrated by an example.The experimental results demonstrate that the probability density distribution of generated data after a large number of network training iterations is consistent with the probability density distribution of monitoring data.The proposed method also effectively demonstrates the generated monitoring data of aircraft engine are in a reasonable range.The method can effectively solve the problem of inaccurate performance degradation evaluation caused by the small amount of aero?engine condition monitoring data.

基于GAN和MS-ResNet的房颤自动检测模型

房颤是一种常见的心律失常疾病,针对现有研究工作大多依赖于单尺度信号段而忽略了不同尺度下潜在的互补信息和数据不平衡问题导致诊断性能下降的问题,提出了一种新颖的基于生成对抗网络(generative adversarial network, GAN)和多尺度残差网络(multiscale residual net, MS-ResNet)的房颤自动检测模型,该网络使用GAN合成具有高形态相似性的单导联心电数据来解决数据的隐私和不平衡问题。同时,设计了MS-ResNet特征提取策略,从不同尺度提取不同大小信号段的特征,从而有效地捕捉P波消失和RR间期不规则特征。该模型联合这两种策略不仅为房颤自动检测生成高质量心电图(electrocardiogram,ECG)数据,还可以利用多尺度网格提取不同波之间的时序特征。在PhysioNet Challenge2017公开ECG数据集上以及平衡后的数据集上评估了MS-ResNet的性能,并将其与现有的房颤分类模型进行了比较。实验结果表明,MS-ResNet在平衡后的数据集上平均F1值和精确率分别达到0.914 1和91.56%,与不平衡数据集相比,F1提高了4.5%,精确率提高了3.5%。

Generative Adversarial Networks:Introduction and Outlook

Recently, generative adversarial networks(GANs)have become a research focus of artificial intelligence. Inspired by two-player zero-sum game, GANs comprise a generator and a discriminator, both trained under the adversarial learning idea.The goal of GANs is to estimate the potential distribution of real data samples and generate new samples from that distribution.Since their initiation, GANs have been widely studied due to their enormous prospect for applications, including image and vision computing, speech and language processing, etc. In this review paper, we summarize the state of the art of GANs and look into the future. Firstly, we survey GANs' proposal background,theoretic and implementation models, and application fields.Then, we discuss GANs' advantages and disadvantages, and their development trends. In particular, we investigate the relation between GANs and parallel intelligence,with the conclusion that GANs have a great potential in parallel systems research in terms of virtual-real interaction and integration. Clearly, GANs can provide substantial algorithmic support for parallel intelligence.

Generative Adversarial Network Based Heuristics for Sampling-Based Path Planning

Sampling-based path planning is a popular methodology for robot path planning.With a uniform sampling strategy to explore the state space,a feasible path can be found without the complex geometric modeling of the configuration space.However,the quality of the initial solution is not guaranteed,and the convergence speed to the optimal solution is slow.In this paper,we present a novel image-based path planning algorithm to overcome these limitations.Specifically,a generative adversarial network(GAN)is designed to take the environment map(denoted as RGB image)as the input without other preprocessing works.The output is also an RGB image where the promising region(where a feasible path probably exists)is segmented.This promising region is utilized as a heuristic to achieve non-uniform sampling for the path planner.We conduct a number of simulation experiments to validate the effectiveness of the proposed method,and the results demonstrate that our method performs much better in terms of the quality of the initial solution and the convergence speed to the optimal solution.Furthermore,apart from the environments similar to the training set,our method also works well on the environments which are very different from the training set.

General image classification method based on semi-supervised generative adversarial networks

Generative adversarial networks(GANs) have become a competitive method among computer vision tasks. There have been many studies devoted to utilizing generative network to do generative tasks, such as images synthesis. In this paper, a semi-supervised learning scheme is incorporated with generative adversarial network on image classification tasks to improve the image classification accuracy. Two applications of GANs are mainly focused on: semi-supervised learning and generation of images which can be as real as possible. The whole process is divided into two sections. First, only a small part of the dataset is utilized as labeled training data. And then a huge amount of samples generated from the generator is added into the training samples to improve the generalization of the discriminator. Through the semi-supervised learning scheme, full use of the unlabeled data is made which may contain potential information. Thus, the classification accuracy of the discriminator can be improved. Experimental results demonstrate the improvement of the classification accuracy of discriminator among different datasets, such as MNIST, CIFAR-10.

Ballistic response of armour plates using Generative Adversarial Networks

It is important to understand how ballistic materials respond to impact from projectiles such that informed decisions can be made in the design process of protective armour systems. Ballistic testing is a standards-based process where materials are tested to determine whether they meet protection, safety and performance criteria. For the V50ballistic test, projectiles are fired at different velocities to determine a key design parameter known as the ballistic limit velocity(BLV), the velocity above which projectiles perforate the target. These tests, however, are destructive by nature and as such there can be considerable associated costs, especially when studying complex armour materials and systems. This study proposes a unique solution to the problem using a recent class of machine learning system known as the Generative Adversarial Network(GAN). The GAN can be used to generate new ballistic samples as opposed to performing additional destructive experiments. A GAN network architecture is tested and trained on three different ballistic data sets, and their performance is compared. The trained networks were able to successfully produce ballistic curves with an overall RMSE of between 10 and 20 % and predicted the V50BLV in each case with an error of less than 5 %. The results demonstrate that it is possible to train generative networks on a limited number of ballistic samples and use the trained network to generate many new samples representative of the data that it was trained on. The paper spotlights the benefits that generative networks can bring to ballistic applications and provides an alternative to expensive testing during the early stages of the design process.

Single Image Dehazing: An Analysis on Generative Adversarial Network

Haze is a very common phenomenon that degrades or reduces visibility. It causes various problems where high-quality images are required such as traffic and security monitoring. So haze removal from scenes is an immediate demand for clear vision. Recently, in addition to the conventional dehazing mechanisms, different types of deep generative adversarial networks (GAN) are applied to suppress the noise and improve the dehazing performance. But it is unclear how these algorithms would perform on hazy images acquired “in the wild” and how we could gauge the progress in the field. To bridge this gap, this presents a comprehensive study on three single image dehazing state-of-the-art GAN models, such as AOD-Net, cGAN, and DHSGAN. We have experimented using benchmark dataset consisting of both synthetic and real-world hazy images. The obtained results are evaluated both quantitatively and qualitatively. Among these techniques, the DHSGAN gives the best performance.

Solar image deconvolution by generative adversarial network

With aperture synthesis(AS)technique,a number of small antennas can be assembled to form a large telescope whose spatial resolution is determined by the distance of two farthest antennas instead of the diameter of a single-dish antenna.In contrast from a direct imaging system,an AS telescope captures the Fourier coefficients of a spatial object,and then implement inverse Fourier transform to reconstruct the spatial image.Due to the limited number of antennas,the Fourier coefficients are extremely sparse in practice,resulting in a very blurry image.To remove/reduce blur,“CLEAN”deconvolution has been widely used in the literature.However,it was initially designed for a point source.For an extended source,like the Sun,its efficiency is unsatisfactory.In this study,a deep neural network,referring to Generative Adversarial Network(GAN),is proposed for solar image deconvolution.The experimental results demonstrate that the proposed model is markedly better than traditional CLEAN on solar images.The main purpose of this work is visual inspection instead of quantitative scientific computation.We believe that this will also help scientists to better understand solar phenomena with high quality images.

Generative Adversarial Network with Separate Learning Rule for Image Generation

Boundary equilibrium generative adversarial networks(BEGANs)are the improved version of generative adversarial networks(GANs).In this paper,an improved BEGAN with a skip-connection technique in the generator and the discriminator is proposed.Moreover,an alternative time-scale update rule is adopted to balance the learning rate of the generator and the discriminator.Finally,the performance of the proposed method is quantitatively evaluated by Fréchet inception distance(FID)and inception score(IS).The test results show that the performance of the proposed method is better than that of the original BEGAN.

Evolution and Effectiveness of Loss Functions in Generative Adversarial Networks

Recently,the evolution of Generative Adversarial Networks(GANs)has embarked on a journey of revolutionizing the field of artificial and computational intelligence.To improve the generating ability of GANs,various loss functions are introduced to measure the degree of similarity between the samples generated by the generator and the real data samples,and the effectiveness of the loss functions in improving the generating ability of GANs.In this paper,we present a detailed survey for the loss functions used in GANs,and provide a critical analysis on the pros and cons of these loss functions.First,the basic theory of GANs along with the training mechanism are introduced.Then,the most commonly used loss functions in GANs are introduced and analyzed.Third,the experimental analyses and comparison of these loss functions are presented in different GAN architectures.Finally,several suggestions on choosing suitable loss functions for image synthesis tasks are given.

Exploration of the Relation between Input Noise and Generated Image in Generative Adversarial Networks

In this paper,we propose a hybrid model aiming to map the input noise vector to the label of the generated image by the generative adversarial network(GAN).This model mainly consists of a pre-trained deep convolution generative adversarial network(DCGAN)and a classifier.By using the model,we visualize the distribution of two-dimensional input noise,leading to a specific type of the generated image after each training epoch of GAN.The visualization reveals the distribution feature of the input noise vector and the performance of the generator.With this feature,we try to build a guided generator(GG)with the ability to produce a fake image we need.Two methods are proposed to build GG.One is the most significant noise(MSN)method,and the other utilizes labeled noise.The MSN method can generate images precisely but with less variations.In contrast,the labeled noise method has more variations but is slightly less stable.Finally,we propose a criterion to measure the performance of the generator,which can be used as a loss function to effectively train the network.

融合情感分析和GAN-TrellisNet的股价预测方法

将时序深度神经网络应用于股票价格预测,已成为量化金融领域的重要研究方向。时序神经网络具有很好的序列数据捕捉能力和学习记忆能力,在股票预测上有一定适用性。但是现有的模型大多存在预测准确度不高、模型结构复杂导致训练时间较长等问题.为了解决以上问题,提出了一种基于情感分析和GAN-TrellisNet的股价预测方法。提出了一个基于LSTM-CNN的情感分析模型,用于分析爬虫获取的主流金融论坛股票评论,并获得股票情感指数。为了提高预测准确度,将情感指数和百度搜索指数加入股票交易数据中作为训练集,提出了一个基于TrellisNet和CNN的改进型GAN股价预测模型,利用TrellisNet生成器的卷积特性来捕捉数据的局部特征,选取特征提取能力较强的CNN作为判别器来区别预测结果和真实股价。通过选取10只代表性股票和三种大盘指数的不同时段数据进行算法验证,结果表明,与ConvLSTM和GAN-LSTM预测模型相比,GAN-TrellisNet模型能有效缩短训练时间,提高预测准确率。

Hybrid tri-memristor hyperchaotic map and application in Wasserstein Generative Adversarial Nets

Inspired by basic circuit connection methods,memristors can also be utilized in the construction of complex discrete chaotic systems.To investigate the dynamical effects of hybrid memristors,we propose two hybrid tri-memristor hyperchaotic(HTMH)mapping structures based on the hybrid parallel/cascade and cascade/parallel operations,respectively.Taking the HTMH mapping structure with hybrid parallel/cascade operation as an example,this map possesses a spatial invariant set whose stability is closely related to the initial states of the memristors.Dynamics distributions and bifurcation behaviours dependent on the control parameters are explored with numerical tools.Specifically,the memristor initial offset-boosting mechanism is theoretically demonstrated,and memristor initial offset-boosting behaviours are numerically verified.The results clarify that the HTMH map can exhibit hyperchaotic behaviours and extreme multistability with homogeneous coexisting infinite attractors.In addition,an FPGA hardware platform is fabricated to implement the HTMH map and generate pseudorandom numbers(PRNs)with high randomness.Notably,the generated PRNs can be applied in Wasserstein generative adversarial nets(WGANs)to enhance training stability and generation capability.

QAR Data Imputation Using Generative Adversarial Network with Self-Attention Mechanism

Quick Access Recorder(QAR),an important device for storing data from various flight parameters,contains a large amount of valuable data and comprehensively records the real state of the airline flight.However,the recorded data have certain missing values due to factors,such as weather and equipment anomalies.These missing values seriously affect the analysis of QAR data by aeronautical engineers,such as airline flight scenario reproduction and airline flight safety status assessment.Therefore,imputing missing values in the QAR data,which can further guarantee the flight safety of airlines,is crucial.QAR data also have multivariate,multiprocess,and temporal features.Therefore,we innovatively propose the imputation models A-AEGAN("A"denotes attention mechanism,"AE"denotes autoencoder,and"GAN"denotes generative adversarial network)and SA-AEGAN("SA"denotes self-attentive mechanism)for missing values of QAR data,which can be effectively applied to QAR data.Specifically,we apply an innovative generative adversarial network to impute missing values from QAR data.The improved gated recurrent unit is then introduced as the neural unit of GAN,which can successfully capture the temporal relationships in QAR data.In addition,we modify the basic structure of GAN by using an autoencoder as the generator and a recurrent neural network as the discriminator.The missing values in the QAR data are imputed by using the adversarial relationship between generator and discriminator.We introduce an attention mechanism in the autoencoder to further improve the capability of the proposed model to capture the features of QAR data.Attention mechanisms can maintain the correlation among QAR data and improve the capability of the model to impute missing data.Furthermore,we improve the proposed model by integrating a self-attention mechanism to further capture the relationship between different parameters within the QAR data.Experimental results on real datasets demonstrate that the model can reasonably impute the missing values in QAR data with excellent results.

基于UNet3+生成对抗网络的视频异常检测

为解决传统视频异常检测方法在不同场景下多尺度特征提取不完全的问题,提出两种方法:一种是用于简单场景的基于UNet3+的生成对抗网络方法(简称U3P^(2)),另一种是用于复杂场景的基于UNet++的生成对抗网络方法(简称UP^(3))。两种方法分别对连续输入的视频帧生成预测,引入多种损失函数和光流模型学习其外观与运动信息,通过计算AUC进行性能评估。U3P^(2)方法以6.3 M参数量在Ped2数据集的AUC提升约0.6%,而UP^(3)方法在Avenue数据集的AUC提升约0.8%,验证其能够有效应对不同场景下的异常检测任务。

MACDCGAN的发电机轴承故障诊断方法

在实际工况中,发电机中传感器采集到的故障样本数据有限,使用基于深度学习的方法进行故障诊断存在过拟合问题导致模型泛化能力较差以及诊断精度不高。为了解决这个问题,采用样本扩充的思路,提出了一种改进的辅助分类器条件深度卷积生成对抗网络(MACDCGAN)的故障诊断方法。通过对采集的一维时序信号进行小波变换增强特征,构建简化结构参数的条件深度卷积生成对抗网络模型生成样本,并在模型中采用Wasserstein距离优化损失函数解决训练过程中存在模式崩塌和梯度消失的缺点;通过添加一个独立的分类器来改进分类模型的兼容性,并在分类器中引入学习率衰减算法增加模型稳定性。试验结果表明,该方法可以有效地提高故障诊断的精度,并且验证了所提模型具有良好的泛化性能。