Automated Video Generation of Moving Digits from Text Using Deep Deconvolutional Generative Adversarial Network

Generating realistic and synthetic video from text is a highly challenging task due to the multitude of issues involved,including digit deformation,noise interference between frames,blurred output,and the need for temporal coherence across frames.In this paper,we propose a novel approach for generating coherent videos of moving digits from textual input using a Deep Deconvolutional Generative Adversarial Network(DD-GAN).The DDGAN comprises a Deep Deconvolutional Neural Network(DDNN)as a Generator(G)and a modified Deep Convolutional Neural Network(DCNN)as a Discriminator(D)to ensure temporal coherence between adjacent frames.The proposed research involves several steps.First,the input text is fed into a Long Short Term Memory(LSTM)based text encoder and then smoothed using Conditioning Augmentation(CA)techniques to enhance the effectiveness of the Generator(G).Next,using a DDNN to generate video frames by incorporating enhanced text and random noise and modifying a DCNN to act as a Discriminator(D),effectively distinguishing between generated and real videos.This research evaluates the quality of the generated videos using standard metrics like Inception Score(IS),Fréchet Inception Distance(FID),Fréchet Inception Distance for video(FID2vid),and Generative Adversarial Metric(GAM),along with a human study based on realism,coherence,and relevance.By conducting experiments on Single-Digit Bouncing MNIST GIFs(SBMG),Two-Digit Bouncing MNIST GIFs(TBMG),and a custom dataset of essential mathematics videos with related text,this research demonstrates significant improvements in both metrics and human study results,confirming the effectiveness of DD-GAN.This research also took the exciting challenge of generating preschool math videos from text,handling complex structures,digits,and symbols,and achieving successful results.The proposed research demonstrates promising results for generating coherent videos from textual input.

Feature-Based Augmentation in Sarcasm Detection Using Reverse Generative Adversarial Network

Sarcasm detection in text data is an increasingly vital area of research due to the prevalence of sarcastic content in online communication.This study addresses challenges associated with small datasets and class imbalances in sarcasm detection by employing comprehensive data pre-processing and Generative Adversial Network(GAN)based augmentation on diverse datasets,including iSarcasm,SemEval-18,and Ghosh.This research offers a novel pipeline for augmenting sarcasm data with Reverse Generative Adversarial Network(RGAN).The proposed RGAN method works by inverting labels between original and synthetic data during the training process.This inversion of labels provides feedback to the generator for generating high-quality data closely resembling the original distribution.Notably,the proposed RGAN model exhibits performance on par with standard GAN,showcasing its robust efficacy in augmenting text data.The exploration of various datasets highlights the nuanced impact of augmentation on model performance,with cautionary insights into maintaining a delicate balance between synthetic and original data.The methodological framework encompasses comprehensive data pre-processing and GAN-based augmentation,with a meticulous comparison against Natural Language Processing Augmentation(NLPAug)as an alternative augmentation technique.Overall,the F1-score of our proposed technique outperforms that of the synonym replacement augmentation technique using NLPAug.The increase in F1-score in experiments using RGAN ranged from 0.066%to 1.054%,and the use of standard GAN resulted in a 2.88%increase in F1-score.The proposed RGAN model outperformed the NLPAug method and demonstrated comparable performance to standard GAN,emphasizing its efficacy in text data augmentation.

Generative Adversarial Networks for Secure Data Transmission in Wireless Network

In this paper,a communication model in cognitive radios is developed and uses machine learning to learn the dynamics of jamming attacks in cognitive radios.It is designed further to make their transmission decision that automati-cally adapts to the transmission dynamics to mitigate the launched jamming attacks.The generative adversarial learning neural network(GALNN)or genera-tive dynamic neural network(GDNN)automatically learns with the synthesized training data(training)with a generator and discriminator type neural networks that encompass minimax game theory.The elimination of the jamming attack is carried out with the assistance of the defense strategies and with an increased detection rate in the generative adversarial network(GAN).The GDNN with game theory is designed to validate the channel condition with the cross entropy loss function and back-propagation algorithm,which improves the communica-tion reliability in the network.The simulation is conducted in NS2.34 tool against several performance metrics to reduce the misdetection rate and false alarm rates.The results show that the GDNN obtains an increased rate of successful transmis-sion by taking optimal actions to act as a defense mechanism to mislead the jam-mer,where the jammer makes high misclassification errors on transmission dynamics.

A Sketch-Based Generation Model for Diverse Ceramic Tile Images Using Generative Adversarial Network

Ceramic tiles are one of the most indispensable materials for interior decoration.The ceramic patterns can’t match the design requirements in terms of diversity and interactivity due to their natural textures.In this paper,we propose a sketch-based generation method for generating diverse ceramic tile images based on a hand-drawn sketches using Generative Adversarial Network(GAN).The generated tile images can be tailored to meet the specific needs of the user for the tile textures.The proposed method consists of four steps.Firstly,a dataset of ceramic tile images with diverse distributions is created and then pre-trained based on GAN.Secondly,for each ceramic tile image in the dataset,the corresponding sketch image is generated and then the mapping relationship between the images is trained based on a sketch extraction network using ResNet Block and jump connection to improve the quality of the generated sketches.Thirdly,the sketch style is redefined according to the characteristics of the ceramic tile images and then double cross-domain adversarial loss functions are employed to guide the ceramic tile generation network for fitting in the direction of the sketch style and to improve the training speed.Finally,we apply hidden space perturbation and interpolation for further enriching the output textures style and satisfying the concept of“one style with multiple faces”.We conduct the training process of the proposed generation network on 2583 ceramic tile images dataset.To measure the generative diversity and quality,we use Frechet Inception Distance(FID)and Blind/Referenceless Image Spatial Quality Evaluator(BRISQUE)metrics.The experimental results prove that the proposed model greatly enhances the generation results of the ceramic tile images,with FID of 32.47 and BRISQUE of 28.44.

MACDCGAN的发电机轴承故障诊断方法

在实际工况中,发电机中传感器采集到的故障样本数据有限,使用基于深度学习的方法进行故障诊断存在过拟合问题导致模型泛化能力较差以及诊断精度不高。为了解决这个问题,采用样本扩充的思路,提出了一种改进的辅助分类器条件深度卷积生成对抗网络(MACDCGAN)的故障诊断方法。通过对采集的一维时序信号进行小波变换增强特征,构建简化结构参数的条件深度卷积生成对抗网络模型生成样本,并在模型中采用Wasserstein距离优化损失函数解决训练过程中存在模式崩塌和梯度消失的缺点;通过添加一个独立的分类器来改进分类模型的兼容性,并在分类器中引入学习率衰减算法增加模型稳定性。试验结果表明,该方法可以有效地提高故障诊断的精度,并且验证了所提模型具有良好的泛化性能。

基于Transformer和GAN的对抗样本生成算法

对抗攻击与防御是计算机安全领域的一个热门研究方向。针对现有基于梯度的对抗样本生成方法可视质量差、基于优化的方法生成效率低的问题,提出基于Transformer和生成对抗网络(GAN)的对抗样本生成算法Trans-GAN。首先利用Transformer强大的视觉表征能力,将其作为重构网络,用于接收干净图像并生成攻击噪声;其次将Transformer重构网络作为生成器,与基于深度卷积网络的鉴别器相结合组成GAN网络架构,提高生成图像的真实性并保证训练的稳定性,同时提出改进的注意力机制Targeted Self-Attention,在训练网络时引入目标标签作为先验知识,指导网络模型学习生成具有特定攻击目标的对抗扰动;最后利用跳转连接将对抗噪声施加在干净样本上,形成对抗样本,攻击目标分类网络。实验结果表明:Trans-GAN算法针对MNIST数据集中2种模型的攻击成功率都达到99.9%以上,针对CIFAR10数据集中2种模型的攻击成功率分别达到96.36%和98.47%,优于目前先进的基于生成式的对抗样本生成方法;相比快速梯度符号法和投影梯度下降法,Trans-GAN算法生成的对抗噪声扰动量更小,形成的对抗样本更加自然,满足人类视觉不易分辨的要求。

融合门控变换机制和GAN的低光照图像增强方法

针对低光照图像增强过程中存在的配对图像数据依赖、细节损失严重和噪声放大问题,提出结合门控通道变换机制和生成对抗网络(GAN)的低光照图像增强方法AGR-GAN,该方法可以在没有低/正常光图像对的情况下进行训练。首先,设计特征提取网络,该网络由多个基于门控通道变换单元的多尺度卷积残差模块构成,以提取输入图像的全局上下文特征和多尺度局部特征信息;然后,在特征融合网络中,采用卷积残差结构将提取的深浅层特征进行充分融合,再引入横向跳跃连接结构,最大程度保留细节特征信息,获得最终的增强图像;最后,引入联合损失函数指导网络训练过程,抑制图像噪声,使增强图像色彩更自然匀称。实验结果表明,该方法在主观视觉分析和客观指标评价方面相较其他算法均具有显著优势,其能有效提高低光照图像的亮度和对比度,减弱图像噪声,增强后的图像更清晰且色彩更真实,峰值信噪比、结构相似度和无参考图像质量评价指标平均可达16.48 dB、0.93和3.37。

基于有效注意力和GAN结合的脑卒中EEG增强算法

在基于脑电的卒中分类诊断任务中,以卷积神经网络为基础的深度模型得到广泛应用,但由于卒中类别病患样本数量少,导致数据集类别不平衡,降低了分类精度。现有的少数类数据增强方法大多采用生成对抗网络(GAN),生成效果一般,虽然可通过引入缩放点乘注意力改善样本生成质量,但存储及运算代价往往较大。针对此问题,构建一种基于线性有效注意力的渐进式数据增强算法LESA-CGAN。首先,算法采用双层自编码条件生成对抗网络架构,分别进行脑电标签特征提取及脑电样本生成,并使生成过程逐层精细化;其次,通过在编码部分引入线性有效自注意力(LESA)模块,加强脑电的标签隐层特征提取,并降低网络整体的运算复杂度。消融与对比实验结果表明,在合理的编码层数与生成数据比例下,LESA-CGAN与其他基准方法相比计算资源占用较少,且在样本生成质量指标上实现了10%的性能提升,各频段生成的脑电特征样本均更加自然,同时将病患分类的准确率和敏感度提高到了98.85%和98.79%。

Generative Adversarial Networks:Introduction and Outlook

Recently, generative adversarial networks(GANs)have become a research focus of artificial intelligence. Inspired by two-player zero-sum game, GANs comprise a generator and a discriminator, both trained under the adversarial learning idea.The goal of GANs is to estimate the potential distribution of real data samples and generate new samples from that distribution.Since their initiation, GANs have been widely studied due to their enormous prospect for applications, including image and vision computing, speech and language processing, etc. In this review paper, we summarize the state of the art of GANs and look into the future. Firstly, we survey GANs' proposal background,theoretic and implementation models, and application fields.Then, we discuss GANs' advantages and disadvantages, and their development trends. In particular, we investigate the relation between GANs and parallel intelligence,with the conclusion that GANs have a great potential in parallel systems research in terms of virtual-real interaction and integration. Clearly, GANs can provide substantial algorithmic support for parallel intelligence.

基于改进MMD-GAN的可再生能源随机场景生成

针对可再生能源出力不确定性的准确表征问题,提出了一种基于改进的最大均值差异生成对抗网络(maximum mean discrepancy generative adversarial networks,MMD-GAN)的可再生能源随机场景生成方法。首先,阐述了GAN及MMD-GAN的基本原理,提出了MMD-GAN的改进方案,即在MMD-GAN的基础上改进鉴别器损失函数,并采用谱归一化和有界高斯核提升生成器和鉴别器的训练稳定性。然后,设计了基于改进MMD-GAN的可再生能源随机场景生成流程。最后,分析了所提方法在可再生能源随机场景生成中的效果,比较了改进MMD-GAN方法与MMD-GAN方法及典型GAN方法的性能差异。结果表明,改进MMD-GAN方法在生成分布和真实分布的Wasserstein距离上较对比方法降低超过50%,生成的场景精度得到有效提升。

Generative Adversarial Network Based Heuristics for Sampling-Based Path Planning

Sampling-based path planning is a popular methodology for robot path planning.With a uniform sampling strategy to explore the state space,a feasible path can be found without the complex geometric modeling of the configuration space.However,the quality of the initial solution is not guaranteed,and the convergence speed to the optimal solution is slow.In this paper,we present a novel image-based path planning algorithm to overcome these limitations.Specifically,a generative adversarial network(GAN)is designed to take the environment map(denoted as RGB image)as the input without other preprocessing works.The output is also an RGB image where the promising region(where a feasible path probably exists)is segmented.This promising region is utilized as a heuristic to achieve non-uniform sampling for the path planner.We conduct a number of simulation experiments to validate the effectiveness of the proposed method,and the results demonstrate that our method performs much better in terms of the quality of the initial solution and the convergence speed to the optimal solution.Furthermore,apart from the environments similar to the training set,our method also works well on the environments which are very different from the training set.

基于Transformer-GAN的农产品包装版式布局智能设计方法

本研究提出一种基于Transformer-GAN的农产品包装版式布局智能设计方法,旨在解决现阶段的农产品包装主要依赖通版包装、缺乏产品特色等问题。首先,设计了内容感知模块,学习包装设计的内容特征;其次,提出一种设计序列模块,对包装布局信息进行序列化处理;最后,融合内容感知和布局信息,使模型学习图像的内容特征和布局特征,输出包装版式布局设计图。与先前的模型相比,本研究模型具有更好的设计性能和可解释性,同时创新性地将布局智能设计方法应用于包装设计领域。实验结果表明,设计序列模块提升了设计的有效性,序列化的布局特征相较于非序列化的特征更能生成优质的布局。该模型具有较强的可解释性,在农产品包装版式设计上具有良好的生成性能。

面向舰船目标检测的SAR图像数据PCGAN生成方法

针对现有合成孔径雷达(SAR)图像数据生成方法大多无法同时生成舰船图像及其检测标签的问题,面向SAR舰船图像生成及目标检测任务,构建基于位置信息的条件生成对抗网络(PCGAN).首先,提出将舰船位置信息作为约束条件用于限制生成图像中舰船的位置,并将其作为舰船图像的检测标签;随后,引入Wasserstein距离稳定PCGAN的训练过程;最后,利用生成的SAR舰船图像及对应检测标签完成YOLOv3网络的端到端训练,实现舰船数据增强与目标检测的协同学习,进而获得更耦合目标检测实际应用的多样性数据.在HRSID(high resolution SAR image dataset)数据集上的实验结果表明,PCGAN方法能生成清晰、鲁棒的SAR舰船数据,舰船检测准确度最高提升1.01%,验证了所提出方法的有效性.

融合情感分析和GAN-TrellisNet的股价预测方法

将时序深度神经网络应用于股票价格预测,已成为量化金融领域的重要研究方向。时序神经网络具有很好的序列数据捕捉能力和学习记忆能力,在股票预测上有一定适用性。但是现有的模型大多存在预测准确度不高、模型结构复杂导致训练时间较长等问题.为了解决以上问题,提出了一种基于情感分析和GAN-TrellisNet的股价预测方法。提出了一个基于LSTM-CNN的情感分析模型,用于分析爬虫获取的主流金融论坛股票评论,并获得股票情感指数。为了提高预测准确度,将情感指数和百度搜索指数加入股票交易数据中作为训练集,提出了一个基于TrellisNet和CNN的改进型GAN股价预测模型,利用TrellisNet生成器的卷积特性来捕捉数据的局部特征,选取特征提取能力较强的CNN作为判别器来区别预测结果和真实股价。通过选取10只代表性股票和三种大盘指数的不同时段数据进行算法验证,结果表明,与ConvLSTM和GAN-LSTM预测模型相比,GAN-TrellisNet模型能有效缩短训练时间,提高预测准确率。

基于GAN改进的红外光与可见光图像融合算法研究

针对夜晚户外场景下,传统的单一鉴别器生成对抗网络(GAN)容易忽略红外光的亮度信息和边缘信息的问题,提出一种基于注意力机制与双鉴别器的红外光与可见光图像融合算法。首先,为了有针对性地获得红外光图像的目标信息和可见光图像的背景纹理信息,在生成器网络中引入通道注意力机制;其次,使用双鉴别器的生成对抗网络,并设计一种新的鉴别器输入,在提高训练稳定性的同时更好地保留源图像信息;最后,损失函数设置为对抗损失、结构相似性损失和梯度损失,以约束鉴别器使其生成细节信息丰富的融合图像。在TNO数据集上的实验结果表明,所提算法得到的融合图像梯度变化更明显、边缘更加清晰,更符合人眼视觉效果。

General image classification method based on semi-supervised generative adversarial networks

Generative adversarial networks(GANs) have become a competitive method among computer vision tasks. There have been many studies devoted to utilizing generative network to do generative tasks, such as images synthesis. In this paper, a semi-supervised learning scheme is incorporated with generative adversarial network on image classification tasks to improve the image classification accuracy. Two applications of GANs are mainly focused on: semi-supervised learning and generation of images which can be as real as possible. The whole process is divided into two sections. First, only a small part of the dataset is utilized as labeled training data. And then a huge amount of samples generated from the generator is added into the training samples to improve the generalization of the discriminator. Through the semi-supervised learning scheme, full use of the unlabeled data is made which may contain potential information. Thus, the classification accuracy of the discriminator can be improved. Experimental results demonstrate the improvement of the classification accuracy of discriminator among different datasets, such as MNIST, CIFAR-10.

Ballistic response of armour plates using Generative Adversarial Networks

It is important to understand how ballistic materials respond to impact from projectiles such that informed decisions can be made in the design process of protective armour systems. Ballistic testing is a standards-based process where materials are tested to determine whether they meet protection, safety and performance criteria. For the V50ballistic test, projectiles are fired at different velocities to determine a key design parameter known as the ballistic limit velocity(BLV), the velocity above which projectiles perforate the target. These tests, however, are destructive by nature and as such there can be considerable associated costs, especially when studying complex armour materials and systems. This study proposes a unique solution to the problem using a recent class of machine learning system known as the Generative Adversarial Network(GAN). The GAN can be used to generate new ballistic samples as opposed to performing additional destructive experiments. A GAN network architecture is tested and trained on three different ballistic data sets, and their performance is compared. The trained networks were able to successfully produce ballistic curves with an overall RMSE of between 10 and 20 % and predicted the V50BLV in each case with an error of less than 5 %. The results demonstrate that it is possible to train generative networks on a limited number of ballistic samples and use the trained network to generate many new samples representative of the data that it was trained on. The paper spotlights the benefits that generative networks can bring to ballistic applications and provides an alternative to expensive testing during the early stages of the design process.

基于SA/WGAN的新能源场景生成方法

随着新能源渗透率逐年提高,其出力的随机性与波动特性难以准确预测,给电力系统的运行、规划和调度提出了严峻的挑战,因此新能源的不确定性建模受到越来越多的关注。为了更有效地获得新能源出力场景中的时序特征,提出了一种基于数据驱动的新能源场景生成方法,通过采用SA/WGAN模型,把自注意力机制和带有梯度惩罚的生成对抗网络判别器结合,构建基于两种模型结合的深度学习模型,有效突显新能源出力场景中时序特性,增强场景生成中非线性拟合能力。算例结果表明,所提模型的新能源生成场景相较于原始WGAN和WGAN-LSTM的场景生成结果,可以有效提高精准度,同时具备了WGAN-GP训练结果稳定和SA计算速度快的优势,更高效地生成与真实新能源场景分布接近的场景。

基于BERT-GAN的SQL注入攻击检测方法研究

针对主流检测方法识别SQL注入攻击面临的检测准确率不高和用于训练的大量真实数据集通常难以获取的问题,提出了一种用于检测SQL注入攻击的BERT-GAN网络模型。通过半监督学习方案来综合对抗样本,采用微调的BERT模型处理未标记数据,利用生成对抗网络区分对抗性和真实示例以及检测恶意和良性样本。实验结果表明,模型在公开管理和标准化数据集sqli,sqliv2和自建的sqli-extend数据集上与基于CNN、LSTM、BERTBase、GAN-Base模型的SQL注入攻击检测方法对比,准确率、精确率、召回率和F1值均有一定程度提高。模型的目标损失函数收敛速度更快说明所提模型具有高效性。

基于MCGAN模型的降水临近预报

在降水临近预报领域中,现有基于深度学习的雷达回波外推方法存在一些缺点.在图像质量方面,预测图像模糊并缺乏小尺度细节,在预测精度方面,降水结果不够准确.本文提出了一个多尺度生成对抗(MCGAN)模型,该模型由多尺度卷积生成器和全卷积的判别器组成.生成器部分采用编码器-解码器架构,主要包含了多尺度卷积块和下采样门控单元. MCGAN模型使用动态时空变异性损失函数在生成对抗网络(GAN)框架下训练,以达到更精准和更清晰的回波强度和分布预测效果.模型的性能在上海市公共雷达数据集上进行了验证,与主流深度学习模型相比,本文所提模型在图像质量评估中的MSE指标上下降了11.15%,在SSIM指标和PSNR指标上分别增加了8.99%、2.95%;在预测精度评估中, CSI指标、POD指标、HSS指标上平均提高了11.92%、15.89%、9.01%,FAR指标平均降低了14.81%.此外,本文通过消融实验证明了MCGAN模型每个部件的作用.