基于扩散模型的多模态引导图像合成系统

A multimodal guided image composition system based on diffusion model

针对电子商务领域中图片背景生成时含义模糊、与前景有重影、不符合现实逻辑等问题,基于近期迅速发展的生成式AI大模型,提出了一种图像合成任务范式框架。该范式同时利用两种模态提示,即文字提示和图片提示,对生成图像进行控制,以保证前景物体的完整性和前后景一致性;同时使用ControlNet插件对生成图像的风格与深度进行控制,规避了背景生成可能出现的大量不规则结果,提高背景生成的质量。消融实验证明了所用模块的必要性。实验结果表明所提范式能够灵活且受控地生成预期图像集合,为电子商务领域的商品图像合成任务提供了便捷有效的解决范式,极大降低了传统方式所带来的时间和人力资源成本,提升了生产效率与灵活性。

Based on the recently rapidly developing generative AI macro-model,a paradigm framework for image composition tasks was proposed to solve the problems of ambiguous meaning,ghosting with the foreground,and unrealistic logic in image background generation in the e-commerce field.This paradigm utilized both text and image prompts to control the generated image,ensuring the integrity of foreground objects and consistency of foreground and background.At the same time,the ControlNet plugin was used to control the generation style and depth,avoiding many irregular results in image background generation and optimizing the effect of background generation.The ablation experiment results demonstrate the necessity of the modules used.The experimental results show that the proposed paradigm can flexibly and controllably generate the expected set of images,providing a convenient and effective solution paradigm for product image composition tasks in the e-commerce field,which greatly reduces the time and human resource costs of the traditional methods,and enhances production efficiency and flexibility.