A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

Research on Restoration of Murals Based on Diffusion Model and Transformer

Due to the limitations of a priori knowledge and convolution operation,the existing image restoration techniques cannot be directly applied to the cultural relics mural restoration,in order to more accurately restore the original appearance of the cultural relics mural images,an image restoration based on the denoising diffusion probability model(Denoising Diffusion Probability Model(DDPM))and the Transformer method.The process involves two steps:in the first step,the damaged mural image is firstly utilized as the condition to generate the noise image,using the time,condition and noise image patch as the inputs to the noise prediction network,capturing the global dependencies in the input sequence through the multi-attentionmechanismof the input sequence and feedforward neural network processing,and designing a long skip connection between the shallow and deep layers in the Transformer blocks between the shallow and deep layers using long skip connections to fuse the feature information of global and local outputs to maintain the overall consistency of the restoration results;In the second step,taking the noisy image as a condition to direct the diffusion model to back sample to generate the restored image.Experiment results show that the PSNR and SSIM of the proposedmethod are improved by 2%to 9%and 1%to 3.3%,respectively,which are compared to the comparison methods.This study proposed synthesizes the advantages of the diffusionmodel and deep learningmodel to make themural restoration results more accurate.

BeFOI: A Novel Method Based on Conditional Diffusion Model for Medical Image Denoising

The progress in medical imaging technology highlights the importance of image quality for effective diagnosis and treatment.Yet,noise during capture and transmission can compromise image accuracy and reliability,complicating clinical decisions.The rising interest in diffusion models has led to their exploration of denoising images.We present Be-FOI(Better Fluoro Images),a weakly supervised model that uses cine images to denoise fluoroscopic images,both DR types.Trained through precise noise estimation and simulation,BeFOI employs Markov chains to denoise using only the fluoroscopic image as guidance.Our tests show that BeFOI outperforms other methods,reducing noise and enhancing clar-ity and diagnostic utility,making it an effective post-processing tool for medical images.

A novel diffusion model considering curvature radius at the bonding interface in a titanium/steel explosive clad plate

This article introduces an element diffusion behavior model for a titanium/steel explosive clad plate characterized by a typical curved interface during the heat-treatment process. A series of heat-treatment experiments were conducted in the temperature range from 750℃ to 950℃, and the effects of heat-treatment parameters on the microstructural evolution and diffusion behavior were investigated by optical microscopy, scanning electron microscopy, X-ray diffraction analysis, and electron-probe microanalysis. Carbon atoms within the steel matrix were observed to diffuse toward the titanium matrix and to aggregate at the bonding interface at 850℃ or lower; in contrast, when the temperature exceeded 850℃, the mutual diffusion of Ti and Fe occurred, along with the diffusion of C atoms, resulting in the for- marion of Ti-Fe intermetallics （Fe2Ti/FeTi）. The diffusion distances of C, Ti, and Fe atoms increased with increasing heating temperature and/or holding time. On the basis of this diffusion behavior, a novel diffusion model was proposed. This model considers the effects of various factors, including the curvature radius of the curved interface, the diffusion coefficient, the heating temperature, and the holding rime. The experimental results show good agreement with the calculated values. The proposed model could clearly provide a general prediction of the elements＇ diffusion at both straight and curved interfaces.

STABILITY OF INNOVATION DIFFUSION MODEL WITH NONLINEAR ACCEPTANCE

In this article, an innovation diffusion model with the nonlinear acceptance is proposed to describe the dynamics of three competing products in a market. It is proved that the model admits a unique positive equilibrium, which is globally stable by excluding the existence of periodic solutions and by using the theory of three dimensional competition systems.

A Numerical Algorithm Based on Quadratic Finite Element for Two-Dimensional Nonlinear Time Fractional Thermal Diffusion Model

In this article,a high-order scheme,which is formulated by combining the quadratic finite element method in space with a second-order time discrete scheme,is developed for looking for the numerical solution of a two-dimensional nonlinear time fractional thermal diffusion model.The time Caputo fractional derivative is approximated by using the L2-1formula,the first-order derivative and nonlinear term are discretized by some second-order approximation formulas,and the quadratic finite element is used to approximate the spatial direction.The error accuracy O(h3+
t2)is obtained,which is verified by the numerical results.

Valuing Credit Default Swap under a double exponential jump diffusion model

This paper discusses the valuation of the Credit Default Swap based on a jump market, in which the asset price of a firm follows a double exponential jump diffusion process, the value of the debt is driven by a geometric Brownian motion, and the default barrier follows a continuous stochastic process. Using the Gaver-Stehfest algorithm and the non-arbitrage asset pricing theory, we give the default probability of the first passage time, and more, derive the price of the Credit Default Swap.

Study of Pre-equilibrium Fission Based on Diffusion Model

In terms of numerical method of Smoluchowski equation the behavior of fission process in diffusion model has been described and analyzed, including the reliance upon time, as well as the deformation parameters at several nuclear temperatures in this paper. The fission rates and the residual probabilities inside the saddle point are calculated for fissile nucleus n＋^238U reaction and un-fissile nucleus p＋^208Pb reaction. The results indicate that there really exists a transient fission process, which means that the pre-equillbrium fission should be taken into account for the fissile nucleus at the high temperature. Oppositely, the pre-equilibrlum fission could be neglected for the un-fissile nucleus. In the certain case the overshooting phenomenon of the fission rates will occur, which is mainly determined by the diffusive current at the saddle point. The higher the temperature is, the more obvious the overshooting phenomenon is. However, the emissions of the light particles accompanying the diffusion process may weaken or vanish the overshooting phenomenon.

STABILITY AND BIFURCATION ANALYSIS OF A DELAYED INNOVATION DIFFUSION MODEL

In this article, a nonlinear mathematical model for innovation diffusion with stage structure which incorporates the evaluation stage （time delay） is proposed. The model is analyzed by considering the effects of external as well as internal influences and other demographic processes such as emigration, intrinsic growth rate, death rate, etc. The asymptotical stability of the various equilibria is investigated. By analyzing the exponential characteristic equation with delay-dependent coefficients obtained through the variational matrix, it is found that Hopf bifurcation occurs when the evaluation period （time delay, T） passes through a critical value. Applying the normal form theory and the center manifold argument, we de- rive the explicit formulas determining the properties of the bifurcating periodic solutions. To illustrate our theoretical analysis, some numerical simulations are also included.

A Fractional Drift Diffusion Model for Organic Semiconductor Devices

Because charge carriers of many organic semiconductors(OSCs)exhibit fractional drift diffusion(Fr-DD)transport properties,the need to develop a Fr-DD model solver becomes more apparent.However,the current research on solving the governing equations of the Fr-DD model is practically nonexistent.In this paper,an iterative solver with high precision is developed to solve both the transient and steady-state Fr-DD model for organic semiconductor devices.The Fr-DD model is composed of two fractionalorder carriers(i.e.,electrons and holes)continuity equations coupled with Poisson’s equation.By treating the current density as constants within each pair of consecutive grid nodes,a linear Caputo’s fractional-order ordinary differential equation(FrODE)can be produced,and its analytic solution gives an approximation to the carrier concentration.The convergence of the solver is guaranteed by implementing a successive over-relaxation(SOR)mechanism on each loop of Gummel’s iteration.Based on our derivations,it can be shown that the Scharfetter–Gummel discretization method is essentially a special case of our scheme.In addition,the consistency and convergence of the two core algorithms are proved,with three numerical examples designed to demonstrate the accuracy and computational performance of this solver.Finally,we validate the Fr-DD model for a steady-state organic field effect transistor(OFET)by fitting the simulated transconductance and output curves to the experimental data.

Indoor measurement method of smoke screen disturbance efficiency based on Gaussian diffusion model

Because of ground clutter wave interf e rence,it is difficult to measure smoke screen disturbance in the field.In this pape r,a kind of indoor measurement method of smoke screen disturbance efficiency ba sed on Gaussian diffusion model is put forward.As a characteristic,the measur ement result of smoke screen area density proves that the indoor measurement met hod of smoke screen disturbance efficiency based on Gaussian diffusion model is fea sible.

Research on the Diffusion Model of Innovation

In this paper, the model of competing technologies diffusion was set up in order to analyze the relationship of competing technologies in diffusion process, and the stability of diffusion model with two kinds of competing technologies was analyzed. It is concluded that dominator can hold competing vantage by suppressing its opponent, and besides, whether competing technology can gain superiority in diffusion process depends on the success rates of technology diffusion when the two kinds of technology match each other in strength.

High-performance diffusion model for inverse design of high Tc superconductors with effective doping and accurate stoichiometry

The pursuit of designing superconductors with high Tc has been a long-standing endeavor.However,the widespread incorporation of doping in high Tc superconductors significantly impacts electronic structure,intricately influencing Tc.The complex interplay between the structural composition and material performance presents a formidable challenge in superconductor design.Based on a novel generative model,diffusion model,and doping adap-tive representation:three-channel matrix,we have designed a high Tc super-conductors inverse design model called Supercon-Diffusion.It has achieved remarkable success in accurately generating chemical formulas for doped high Tc superconductors.Supercon-Diffusion is capable of generating superconduc-tors that exhibit high Tc and excels at identifying the optimal doping ratios that yield the peak Tc.The doping effectiveness(55%)and electrical neutrality(55%)of the generated doped superconductors exceed those of traditional GAN models by more than tenfold.Density of state calculations on the structures further confirm the validity of the generated superconductors.Additionally,we have proposed 200 potential high Tc superconductors that have not been documented yet.This groundbreaking contribution effectively reduces the search space for high Tc superconductors.Moreover,it successfully establishes a bridge between the interrelated aspects of composition,structure,and prop-erty in superconductors,providing a novel solution for designing other doped materials.

A multilayer network diffusion-based model for reviewer recommendation

With the rapid growth of manuscript submissions,finding eligible reviewers for every submission has become a heavy task.Recommender systems are powerful tools developed in computer science and information science to deal with this problem.However,most existing approaches resort to text mining techniques to match manuscripts with potential reviewers,which require high-quality textual information to perform well.In this paper,we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network,with no requirement for textual information.The network incorporates the relationship of scholar-paper pairs,the collaboration among scholars,and the bibliographic coupling among papers.Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing,with improvements of over 7.62%in recall,5.66%in hit rate,and 47.53%in ranking score.Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem,which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

A Survey of Multimodal Controllable Diffusion Models

Diffusion models have recently emerged as powerful generative models,producing high-fidelity samples across domains.Despite this,they have two key challenges,including improving the time-consuming iterative generation process and controlling and steering the generation process.Existing surveys provide broad overviews of diffusion model advancements.However,they lack comprehensive coverage specifically centered on techniques for controllable generation.This survey seeks to address this gap by providing a comprehensive and coherent review on controllable generation in diffusion models.We provide a detailed taxonomy defining controlled generation for diffusion models.Controllable generation is categorized based on the formulation,methodologies,and evaluation metrics.By enumerating the range of methods researchers have developed for enhanced control,we aim to establish controllable diffusion generation as a distinct subfield warranting dedicated focus.With this survey,we contextualize recent results,provide the dedicated treatment of controllable diffusion model generation,and outline limitations and future directions.To demonstrate applicability,we highlight controllable diffusion techniques for major computer vision tasks application.By consolidating methods and applications for controllable diffusion models,we hope to catalyze further innovations in reliable and scalable controllable generation.

TendiffPure:a convolutional tensor-train denoising diffusion model for purification

Diffusion models are effective purification methods,where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks.However,the efficiency of diffusion models is still a concern,and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps.Hence,we propose TendiffPure as a tensorized and compressed diffusion model for purification.Unlike the knowledge distillation methods,we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition,which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images.The space complexity is reduced from O(N^(2))to O(NR^(2))with R≤4 as the tensor-train rank and N as the number of channels.Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10,Fashion-MNIST,and MNIST datasets for two noises and one adversarial attack.

DiffMat:Latent diffusion models for image-guided material generation

Creating realistic materials is essential in the construction of immersive virtual environments.While existing techniques for material capture and conditional generation rely on flash-lit photos,they often produce artifacts when the illumination mismatches the training data.In this study,we introduce DiffMat,a novel diffusion model that integrates the CLIP image encoder and a multi-layer,crossattention denoising backbone to generate latent materials from images under various illuminations.Using a pre-trained StyleGAN-based material generator,our method converts these latent materials into high-resolution SVBRDF textures,a process that enables a seamless fit into the standard physically based rendering pipeline,reducing the requirements for vast computational resources and expansive datasets.DiffMat surpasses existing generative methods in terms of material quality and variety,and shows adaptability to a broader spectrum of lighting conditions in reference images.

Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

Numerical Simulation of Diffusion Type Traffic Flow Model Using Second-Order Lax-Wendroff Scheme Based on Exponential Velocity Density Function

In order to control traffic congestion, many mathematical models have been used for several decades. In this paper, we study diffusion-type traffic flow model based on exponential velocity density relation, which provides a non-linear second-order parabolic partial differential equation. The analytical solution of the diffusion-type traffic flow model is very complicated to approximate the initial density of the Cauchy problem as a function of x from given data and it may cause a huge error. For the complexity of the analytical solution, the numerical solution is performed by implementing an explicit upwind, explicitly centered, and second-order Lax-Wendroff scheme for the numerical solution. From the comparison of relative error among these three schemes, it is observed that Lax-Wendroff scheme gives less error than the explicit upwind and explicit centered difference scheme. The numerical, analytical analysis and comparative result discussion bring out the fact that the Lax-Wendroff scheme with exponential velocity-density relation of diffusion type traffic flow model is suitable for the congested area and shows a better fit in traffic-congested regions.

An innovation diffusion model for consumer durables with three parameters

Strategic innovation diffusion converts newly created knowledge into increasing a firm’s value primarily through innovative product offerings.In this paper,we present a time-based adoption pattern with pricing and promotional expenditure as a three-dimensional innovation diffusion model(3D-IDM).In our proposed 3D-IDM,we assume that value of the product plays a crucial role of being the major driver of diffusion,and is classified into the following three main factors:(1)continuation time of the product in the market–representing goodwill of the product;(2)price of the product–indicating consumers’buying behaviour;and(3)marketing efforts of the firm.A special form of the Cobb–Douglas production function is used to design the three-dimensional framework.An empirical study is performed on number of consumer-durable sales data to validate and compare the proposed model.Various performance measures are treated uniquely using the Mahalanobis distance-based approach(DBA)to determine the relative strength of each model.