摘要
对文本生成图像任务进行综合评估和整理,根据生成图像的理念,将文本生成图像任务分为3大类:基于生成对抗网络架构生成图像、基于自回归模型架构生成图像、基于扩散模型架构生成图像.针对基于生成对抗网络架构的文本生成图像方法,按照改进的不同技术点归纳为6小类:采用多层次体系嵌套架构、注意力机制的应用、应用孪生网络、采用循环一致方法、深度融合文本特征和改进无条件模型.通过对不同方法的分析,总结并讨论了现有的文本生成图像方法通用评估指标和数据集.
A comprehensive evaluation and categorization of text-to-image generation tasks were conducted.Text-to-image generation tasks were classified into three major categories based on the principles of image generation:text-to-image generation based on the generative adversarial network architecture,text-to-image generation based on the autoregressive model architecture,and text-to-image generation based on the diffusion model architecture.Improvements in different aspects were categorized into six subcategories for text-to-image generation methods based on the generative adversarial network architecture:adoption of multi-level hierarchical architectures,application of attention mechanisms,utilization of siamese networks,incorporation of cycle-consistency methods,deep fusion of text features,and enhancement of unconditional models.The general evaluation indicators and datasets of existing text-to-image methods were summarized and discussed through the analysis of different methods.
作者
曹寅
秦俊平
马千里
孙昊
闫凯
王磊
任家琪
CAO Yin;QIN Junping;MA Qianli;SUN Hao;YAN Kai;WANG Lei;REN Jiaqi(College of Data Science and Applications,Inner Mongolia University of Technology,Hohhot 010000,China;Inner Mongolia Autonomous Region Engineering Technology Research Center of Big Data Based Software Service,Hohhot 010000,China)
出处
《浙江大学学报(工学版)》
EI
CAS
CSCD
北大核心
2024年第2期219-238,共20页
Journal of Zhejiang University:Engineering Science
基金
国家自然科学基金资助项目(61962044)
内蒙自然科学基金资助项目(2019MS06005)
内蒙古自治区科技重大专项项目(2021ZD0015)
自治区直属高校基本科研业务费资助项目(JY20220327)。