A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

BeFOI: A Novel Method Based on Conditional Diffusion Model for Medical Image Denoising

The progress in medical imaging technology highlights the importance of image quality for effective diagnosis and treatment.Yet,noise during capture and transmission can compromise image accuracy and reliability,complicating clinical decisions.The rising interest in diffusion models has led to their exploration of denoising images.We present Be-FOI(Better Fluoro Images),a weakly supervised model that uses cine images to denoise fluoroscopic images,both DR types.Trained through precise noise estimation and simulation,BeFOI employs Markov chains to denoise using only the fluoroscopic image as guidance.Our tests show that BeFOI outperforms other methods,reducing noise and enhancing clar-ity and diagnostic utility,making it an effective post-processing tool for medical images.

A novel diffusion model considering curvature radius at the bonding interface in a titanium/steel explosive clad plate

This article introduces an element diffusion behavior model for a titanium/steel explosive clad plate characterized by a typical curved interface during the heat-treatment process. A series of heat-treatment experiments were conducted in the temperature range from 750℃ to 950℃, and the effects of heat-treatment parameters on the microstructural evolution and diffusion behavior were investigated by optical microscopy, scanning electron microscopy, X-ray diffraction analysis, and electron-probe microanalysis. Carbon atoms within the steel matrix were observed to diffuse toward the titanium matrix and to aggregate at the bonding interface at 850℃ or lower; in contrast, when the temperature exceeded 850℃, the mutual diffusion of Ti and Fe occurred, along with the diffusion of C atoms, resulting in the for- marion of Ti-Fe intermetallics （Fe2Ti/FeTi）. The diffusion distances of C, Ti, and Fe atoms increased with increasing heating temperature and/or holding time. On the basis of this diffusion behavior, a novel diffusion model was proposed. This model considers the effects of various factors, including the curvature radius of the curved interface, the diffusion coefficient, the heating temperature, and the holding rime. The experimental results show good agreement with the calculated values. The proposed model could clearly provide a general prediction of the elements＇ diffusion at both straight and curved interfaces.

STABILITY OF INNOVATION DIFFUSION MODEL WITH NONLINEAR ACCEPTANCE

In this article, an innovation diffusion model with the nonlinear acceptance is proposed to describe the dynamics of three competing products in a market. It is proved that the model admits a unique positive equilibrium, which is globally stable by excluding the existence of periodic solutions and by using the theory of three dimensional competition systems.

A Numerical Algorithm Based on Quadratic Finite Element for Two-Dimensional Nonlinear Time Fractional Thermal Diffusion Model

In this article,a high-order scheme,which is formulated by combining the quadratic finite element method in space with a second-order time discrete scheme,is developed for looking for the numerical solution of a two-dimensional nonlinear time fractional thermal diffusion model.The time Caputo fractional derivative is approximated by using the L2-1formula,the first-order derivative and nonlinear term are discretized by some second-order approximation formulas,and the quadratic finite element is used to approximate the spatial direction.The error accuracy O(h3+
t2)is obtained,which is verified by the numerical results.

STABILITY AND BIFURCATION ANALYSIS OF A DELAYED INNOVATION DIFFUSION MODEL

In this article, a nonlinear mathematical model for innovation diffusion with stage structure which incorporates the evaluation stage （time delay） is proposed. The model is analyzed by considering the effects of external as well as internal influences and other demographic processes such as emigration, intrinsic growth rate, death rate, etc. The asymptotical stability of the various equilibria is investigated. By analyzing the exponential characteristic equation with delay-dependent coefficients obtained through the variational matrix, it is found that Hopf bifurcation occurs when the evaluation period （time delay, T） passes through a critical value. Applying the normal form theory and the center manifold argument, we de- rive the explicit formulas determining the properties of the bifurcating periodic solutions. To illustrate our theoretical analysis, some numerical simulations are also included.

Valuing Credit Default Swap under a double exponential jump diffusion model

This paper discusses the valuation of the Credit Default Swap based on a jump market, in which the asset price of a firm follows a double exponential jump diffusion process, the value of the debt is driven by a geometric Brownian motion, and the default barrier follows a continuous stochastic process. Using the Gaver-Stehfest algorithm and the non-arbitrage asset pricing theory, we give the default probability of the first passage time, and more, derive the price of the Credit Default Swap.

A Fractional Drift Diffusion Model for Organic Semiconductor Devices

Because charge carriers of many organic semiconductors(OSCs)exhibit fractional drift diffusion(Fr-DD)transport properties,the need to develop a Fr-DD model solver becomes more apparent.However,the current research on solving the governing equations of the Fr-DD model is practically nonexistent.In this paper,an iterative solver with high precision is developed to solve both the transient and steady-state Fr-DD model for organic semiconductor devices.The Fr-DD model is composed of two fractionalorder carriers(i.e.,electrons and holes)continuity equations coupled with Poisson’s equation.By treating the current density as constants within each pair of consecutive grid nodes,a linear Caputo’s fractional-order ordinary differential equation(FrODE)can be produced,and its analytic solution gives an approximation to the carrier concentration.The convergence of the solver is guaranteed by implementing a successive over-relaxation(SOR)mechanism on each loop of Gummel’s iteration.Based on our derivations,it can be shown that the Scharfetter–Gummel discretization method is essentially a special case of our scheme.In addition,the consistency and convergence of the two core algorithms are proved,with three numerical examples designed to demonstrate the accuracy and computational performance of this solver.Finally,we validate the Fr-DD model for a steady-state organic field effect transistor(OFET)by fitting the simulated transconductance and output curves to the experimental data.

A multilayer network diffusion-based model for reviewer recommendation

With the rapid growth of manuscript submissions,finding eligible reviewers for every submission has become a heavy task.Recommender systems are powerful tools developed in computer science and information science to deal with this problem.However,most existing approaches resort to text mining techniques to match manuscripts with potential reviewers,which require high-quality textual information to perform well.In this paper,we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network,with no requirement for textual information.The network incorporates the relationship of scholar-paper pairs,the collaboration among scholars,and the bibliographic coupling among papers.Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing,with improvements of over 7.62%in recall,5.66%in hit rate,and 47.53%in ranking score.Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem,which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

TendiffPure:a convolutional tensor-train denoising diffusion model for purification

Diffusion models are effective purification methods,where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks.However,the efficiency of diffusion models is still a concern,and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps.Hence,we propose TendiffPure as a tensorized and compressed diffusion model for purification.Unlike the knowledge distillation methods,we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition,which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images.The space complexity is reduced from O(N^(2))to O(NR^(2))with R≤4 as the tensor-train rank and N as the number of channels.Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10,Fashion-MNIST,and MNIST datasets for two noises and one adversarial attack.

Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

Numerical Simulation of Diffusion Type Traffic Flow Model Using Second-Order Lax-Wendroff Scheme Based on Exponential Velocity Density Function

In order to control traffic congestion, many mathematical models have been used for several decades. In this paper, we study diffusion-type traffic flow model based on exponential velocity density relation, which provides a non-linear second-order parabolic partial differential equation. The analytical solution of the diffusion-type traffic flow model is very complicated to approximate the initial density of the Cauchy problem as a function of x from given data and it may cause a huge error. For the complexity of the analytical solution, the numerical solution is performed by implementing an explicit upwind, explicitly centered, and second-order Lax-Wendroff scheme for the numerical solution. From the comparison of relative error among these three schemes, it is observed that Lax-Wendroff scheme gives less error than the explicit upwind and explicit centered difference scheme. The numerical, analytical analysis and comparative result discussion bring out the fact that the Lax-Wendroff scheme with exponential velocity-density relation of diffusion type traffic flow model is suitable for the congested area and shows a better fit in traffic-congested regions.

An innovation diffusion model for consumer durables with three parameters

Strategic innovation diffusion converts newly created knowledge into increasing a firm’s value primarily through innovative product offerings.In this paper,we present a time-based adoption pattern with pricing and promotional expenditure as a three-dimensional innovation diffusion model(3D-IDM).In our proposed 3D-IDM,we assume that value of the product plays a crucial role of being the major driver of diffusion,and is classified into the following three main factors:(1)continuation time of the product in the market–representing goodwill of the product;(2)price of the product–indicating consumers’buying behaviour;and(3)marketing efforts of the firm.A special form of the Cobb–Douglas production function is used to design the three-dimensional framework.An empirical study is performed on number of consumer-durable sales data to validate and compare the proposed model.Various performance measures are treated uniquely using the Mahalanobis distance-based approach(DBA)to determine the relative strength of each model.

Modeling of Diffusion Transport through Oral Biofilms with the Inverse Problem Method

Aim The purpose of this study was to develop a mathe-matical model to quantitatively describe the passive trans-port of macromolecules within dental biofilms. Methodology Fluorescently labeled dextrans with different molecular mass （3 kD,10 kD,40 kD,70 kD,2 000 kD） were used as a series of diffusion probes. Streptococcus mutans,Streptococcus sanguinis,Actinomyces naeslundii and Fusobacterium nucleatum were used as inocula for biofilm formation. The diffusion processes of different probes through the in vitro biofilm were recorded with a confocal laser microscope. Results Mathematical function of biofilm penetration was constructed on the basis of the inverse problem method. Based on this function,not only the relationship between average concentration of steady-state and molecule weights can be analyzed,but also that between penetrative time and molecule weights. Conclusion This can be used to predict the effective concentration and the penetrative time of anti-biofilm medicines that can diffuse through oral biofilm. Further-more,an improved model for large molecule is proposed by considering the exchange time at the upper boundary of the dental biofilm.

Diffusion analysis and modeling of kinetic behavior for treatment of brine water using electrodialysis process

In this study,the removal of monovalent and divalent cations,Nat,Kt,Mg2t,and Ca2t,in a diluted solution from Chott-El Jerid Lake,Tunisia,was investigated with the electrodialysis technique.The process was tested using two cation-exchange membranes:sulfonated polyether sulfone cross-linked with 10%hexamethylenediamine(HEXCl)and sulfonated polyether sulfone grafted with octylamine(S-PESOS).The commercially available membrane Nafion®was used for comparison.The results showed that Nafion®and S-PESOS membranes had similar removal behaviors,and the investigated cations were ranked in the following descending order in terms of their demineralization rates:Nat>Ca2t>Mg2t>Kt.Divalent cations were more effectively removed by HEXCl than by monovalent cations.The plots based on the WebereMorris model showed a strong linearity.This reveals that intra-particle diffusion was not the removal rate-determining step,and the removal process was controlled by two or more concurrent mechanisms.The Boyd plots did not pass through their origin,and the sole controlling step was determined by film-diffusion resistance,especially after a long period of electrodialysis.Additionally,a semi-empirical model was established to simulate the temporal variation of the treatment process,and the physical significance and values of model parameters were compared for the three membranes.The findings of this study indicate that HEXCl and S-PESOS membranes can be efficiently utilized for water softening,especially when effluents are highly loaded with calcium and magnesium ions.

Convection–Diffusion Model for the Prediction of Anthropogenically-Initiated Wildfire Ignition

A spatial interaction model to predict anthropogenically-initiated accidental and incendiary wildfire ignition probability is developed using fluid flow analogies for human movement patterns. Urban areas with large populations are identified as the sites of global influencing factors, and are modeled as the gravity term. The transportation corridors are identified as local influencing factors, and are modeled using fluid flow analogy as diffusion and convection terms. The model is implemented in ArcGIS, and applied for the prediction of wildfire hazard distribution in southeastern Mississippi. The model shows87 % correlation with historic data in the winter season,whereas the previously developed gravity model shows only 75 % correlation. The normalized error for convection–diffusion model predictions is about 5 % in the winter season, whereas the gravity model shows an error of 7 %.The proposed model is robust as it couples a multi-criteria behavioral pattern within a single dynamic equation to enhance predictive capability. At the same time, the proposed model is more costly than the gravity model as it requires evaluation of distance from intermodal transportation corridors, transportation corridor density, and traffic volume maps. Nonetheless, the model is developed in a modular fashion, such that either global or local terms can be neglected if required.

A Diffusion Model for Optimal Dividend Payment and Risk Control for a Firm under Consideration of the Time Value of Ruin

In this paper,we investigate a model for an insurance company with constraint on risk control.The objective of the insurer is to find a business policy and a dividend payment scheme so as to maximize the expected discounted value of dividend payment,and the expected present value of an amount which the insurer earns until the time of ruin.By solving the constrained Hamilton-Jacobi-Bellman equation,we obtain the explicit expression for value function and the corresponding optimal strategies.

A Specification Test of Stochastic Diffusion Models

In this paper, we propose a hypothesis testing approach to checking model mis-specification in continuous-time stochastic diffusion model. The key idea behind the development of our test statistic is rooted in the generalized information equality in the context of martingale estimating equations. We propose a bootstrap resampling method to implement numerically the proposed diagnostic procedure. Through intensive simulation studies, we show that our approach is well performed in the aspects of type I error control, power improvement as well as computational efficiency.

Study on Opioids Diffusion Based on Improved SIR Model

The abuse of opioids is harmful to the national economy and health.The U.S.government has spent a lot of time,energy and money to deal with this phenomenon.Based on the topic background and team discussion,we deeply excavated the data and information provided in the topic,determined the current use of opioids,and constructed an improved SIR model to determine the source of drug abuse,the mechanism of drug abuse diffusion and the origin of each state through reverse derivation,which provided guidance for the government in the context of opioid abuse.Based on the above results,we simulated and analyzed the improved SIR model and determined the accuracy and stability of the model in the data set.

Dynamics of the HBV model with diffusion and time delay

Chronic hepatitis B infection is a major health problem,with approximately 350 million virus carriers worldwide.In Africa,about 30%-60% of children and 60%-100% of adults have