A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

A multilayer network diffusion-based model for reviewer recommendation

With the rapid growth of manuscript submissions,finding eligible reviewers for every submission has become a heavy task.Recommender systems are powerful tools developed in computer science and information science to deal with this problem.However,most existing approaches resort to text mining techniques to match manuscripts with potential reviewers,which require high-quality textual information to perform well.In this paper,we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network,with no requirement for textual information.The network incorporates the relationship of scholar-paper pairs,the collaboration among scholars,and the bibliographic coupling among papers.Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing,with improvements of over 7.62%in recall,5.66%in hit rate,and 47.53%in ranking score.Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem,which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

BeFOI: A Novel Method Based on Conditional Diffusion Model for Medical Image Denoising

The progress in medical imaging technology highlights the importance of image quality for effective diagnosis and treatment.Yet,noise during capture and transmission can compromise image accuracy and reliability,complicating clinical decisions.The rising interest in diffusion models has led to their exploration of denoising images.We present Be-FOI(Better Fluoro Images),a weakly supervised model that uses cine images to denoise fluoroscopic images,both DR types.Trained through precise noise estimation and simulation,BeFOI employs Markov chains to denoise using only the fluoroscopic image as guidance.Our tests show that BeFOI outperforms other methods,reducing noise and enhancing clar-ity and diagnostic utility,making it an effective post-processing tool for medical images.

A Specification Test of Stochastic Diffusion Models

In this paper, we propose a hypothesis testing approach to checking model mis-specification in continuous-time stochastic diffusion model. The key idea behind the development of our test statistic is rooted in the generalized information equality in the context of martingale estimating equations. We propose a bootstrap resampling method to implement numerically the proposed diagnostic procedure. Through intensive simulation studies, we show that our approach is well performed in the aspects of type I error control, power improvement as well as computational efficiency.

Numerical Simulation of Diffusion Type Traffic Flow Model Using Second-Order Lax-Wendroff Scheme Based on Exponential Velocity Density Function

In order to control traffic congestion, many mathematical models have been used for several decades. In this paper, we study diffusion-type traffic flow model based on exponential velocity density relation, which provides a non-linear second-order parabolic partial differential equation. The analytical solution of the diffusion-type traffic flow model is very complicated to approximate the initial density of the Cauchy problem as a function of x from given data and it may cause a huge error. For the complexity of the analytical solution, the numerical solution is performed by implementing an explicit upwind, explicitly centered, and second-order Lax-Wendroff scheme for the numerical solution. From the comparison of relative error among these three schemes, it is observed that Lax-Wendroff scheme gives less error than the explicit upwind and explicit centered difference scheme. The numerical, analytical analysis and comparative result discussion bring out the fact that the Lax-Wendroff scheme with exponential velocity-density relation of diffusion type traffic flow model is suitable for the congested area and shows a better fit in traffic-congested regions.

Diffusionmodels for time-series applications: a survey

Diffusion models, a family of generative models based on deep learning, have become increasinglyprominent in cutting-edge machine learning research. With distinguished performance in generating samples thatresemble the observed data, diffusion models are widely used in image, video, and text synthesis nowadays. Inrecent years, the concept of diffusion has been extended to time-series applications, and many powerful models havebeen developed. Considering the deficiency of a methodical summary and discourse on these models, we providethis survey as an elementary resource for new researchers in this area and to provide inspiration to motivate futureresearch. For better understanding, we include an introduction about the basics of diffusion models. Except forthis, we primarily focus on diffusion-based methods for time-series forecasting, imputation, and generation, andpresent them, separately, in three individual sections. We also compare different methods for the same applicationand highlight their connections if applicable. Finally, we conclude with the common limitation of diffusion-basedmethods and highlight potential future research directions.

A novel diffusion model considering curvature radius at the bonding interface in a titanium/steel explosive clad plate

This article introduces an element diffusion behavior model for a titanium/steel explosive clad plate characterized by a typical curved interface during the heat-treatment process. A series of heat-treatment experiments were conducted in the temperature range from 750℃ to 950℃, and the effects of heat-treatment parameters on the microstructural evolution and diffusion behavior were investigated by optical microscopy, scanning electron microscopy, X-ray diffraction analysis, and electron-probe microanalysis. Carbon atoms within the steel matrix were observed to diffuse toward the titanium matrix and to aggregate at the bonding interface at 850℃ or lower; in contrast, when the temperature exceeded 850℃, the mutual diffusion of Ti and Fe occurred, along with the diffusion of C atoms, resulting in the for- marion of Ti-Fe intermetallics （Fe2Ti/FeTi）. The diffusion distances of C, Ti, and Fe atoms increased with increasing heating temperature and/or holding time. On the basis of this diffusion behavior, a novel diffusion model was proposed. This model considers the effects of various factors, including the curvature radius of the curved interface, the diffusion coefficient, the heating temperature, and the holding rime. The experimental results show good agreement with the calculated values. The proposed model could clearly provide a general prediction of the elements＇ diffusion at both straight and curved interfaces.

A rapid and accurate direct measurement method of underground coal seam gas content based on dynamic diffusion theory

Coal seam gas content is frequently measured in quantity during underground coal mining operation and coalbed methane(CBM)exploration as a significant basic parameter.Due to the calculation error of lost gas and residual gas in the direct method,the efficiency and accuracy of the current methods are not inadequate to the large area multi-point measurement of coal seam gas content.This paper firstly deduces a simplified theoretical dynamic model for calculating lost gas based on gas dynamic diffusion theory.Secondly,the effects of various factors on gas dynamic diffusion from coal particle are experimentally studied.And sampling procedure of representative coal particle is improved.Thirdly,a new estimation method of residual gas content based on excess adsorption and competitive adsorption theory is proposed.The results showed that the maximum error of calculating the losing gas content by using the new simplified model is only 4%.Considering the influence of particle size on gas diffusion law,the particle size of the collected coal sample is below 0.25 mm,which improves the measurement speed and reflects the safety representativeness of the sample.The determination time of gas content reduced from 36 to 3 h/piece.Moreover,the absolute error is 0.15–0.50 m^3/t,and the relative error is within 5%.A new engineering method for determining the coal seam gas content is developed according to the above research.

STABILITY OF INNOVATION DIFFUSION MODEL WITH NONLINEAR ACCEPTANCE

In this article, an innovation diffusion model with the nonlinear acceptance is proposed to describe the dynamics of three competing products in a market. It is proved that the model admits a unique positive equilibrium, which is globally stable by excluding the existence of periodic solutions and by using the theory of three dimensional competition systems.

A Numerical Algorithm Based on Quadratic Finite Element for Two-Dimensional Nonlinear Time Fractional Thermal Diffusion Model

In this article,a high-order scheme,which is formulated by combining the quadratic finite element method in space with a second-order time discrete scheme,is developed for looking for the numerical solution of a two-dimensional nonlinear time fractional thermal diffusion model.The time Caputo fractional derivative is approximated by using the L2-1formula,the first-order derivative and nonlinear term are discretized by some second-order approximation formulas,and the quadratic finite element is used to approximate the spatial direction.The error accuracy O(h3+
t2)is obtained,which is verified by the numerical results.

Anisotropic diffusion of volatile pollutants at air-water interface

The volatile pollutants that spill into natural waters cause water pollution. Air pollution arises from the water pollution because of volatilization. Mass exchange caused by turbulent fluctuation is stronger in the direction normal to the air-water interface than in other directions due to the large density difference between water and air. In order to explore the characteristics of anisotropic diffusion of the volatile pollutants at the air-water interface, the relationship between velocity gradient and mass transfer rate was established to calculate the turbulent mass diffusivity. A second-order accurate smooth transition differencing scheme （STDS） was proposed to guarantee the boundedness for the flow and mass transfer at the air-water interface. Simulations and experiments were performed to study the trichloroethylene （C2HC13） release. By comparing the anisotropic coupling diffusion model, isotropic coupling diffusion model, and non-coupling diffusion model, the features of the transport of volatile pollutants at the air-water interface were determined. The results show that the anisotropic coupling diffusion model is more accurate than the isotropic coupling diffusion model and non-coupling diffusion model. Mass transfer significantly increases with the increase of the air-water relative velocity at a low relative velocity. However, at a higher relative velocity, an increase in the relative velocity has no effect on mass transfer.

Modeling of Diffusion Transport through Oral Biofilms with the Inverse Problem Method

Aim The purpose of this study was to develop a mathe-matical model to quantitatively describe the passive trans-port of macromolecules within dental biofilms. Methodology Fluorescently labeled dextrans with different molecular mass （3 kD,10 kD,40 kD,70 kD,2 000 kD） were used as a series of diffusion probes. Streptococcus mutans,Streptococcus sanguinis,Actinomyces naeslundii and Fusobacterium nucleatum were used as inocula for biofilm formation. The diffusion processes of different probes through the in vitro biofilm were recorded with a confocal laser microscope. Results Mathematical function of biofilm penetration was constructed on the basis of the inverse problem method. Based on this function,not only the relationship between average concentration of steady-state and molecule weights can be analyzed,but also that between penetrative time and molecule weights. Conclusion This can be used to predict the effective concentration and the penetrative time of anti-biofilm medicines that can diffuse through oral biofilm. Further-more,an improved model for large molecule is proposed by considering the exchange time at the upper boundary of the dental biofilm.

STABILITY AND BIFURCATION ANALYSIS OF A DELAYED INNOVATION DIFFUSION MODEL

In this article, a nonlinear mathematical model for innovation diffusion with stage structure which incorporates the evaluation stage （time delay） is proposed. The model is analyzed by considering the effects of external as well as internal influences and other demographic processes such as emigration, intrinsic growth rate, death rate, etc. The asymptotical stability of the various equilibria is investigated. By analyzing the exponential characteristic equation with delay-dependent coefficients obtained through the variational matrix, it is found that Hopf bifurcation occurs when the evaluation period （time delay, T） passes through a critical value. Applying the normal form theory and the center manifold argument, we de- rive the explicit formulas determining the properties of the bifurcating periodic solutions. To illustrate our theoretical analysis, some numerical simulations are also included.

Simulation research on monomer agglomeration of nonmetallic inclusions in steel with a diffusion limited aggregation model

The monomer agglomeration of nonmetallic inclusions was simulated with a diffusion limited aggregation （DLA） model of the fractal theory. The simulation study with a random two-dimensional diffusion was carried out. The results indicate that the DLA model can be used for the simulation of agglomeration behavior of the cluster-type inclusions. The morphology of clusters was observed with SEM and compared with the simulated agglomerates. The modelling procedure of the DLA model is applicable for the agglomeration process. The uncertainty of agglomeration process and the persuasive average agglomerative ratio was analyzed. The factors about the agglomerative ratio with the collision path distance and the size of particles or seed were discussed. The adherence of the nonmetallic inclusions on the dam, the weir and the walls of a tundish, and the absorption of inclusions by stopper or nozzle were also discussed.

Dynamics of the HBV model with diffusion and time delay

Chronic hepatitis B infection is a major health problem,with approximately 350 million virus carriers worldwide.In Africa,about 30%-60% of children and 60%-100% of adults have

Valuing Credit Default Swap under a double exponential jump diffusion model

This paper discusses the valuation of the Credit Default Swap based on a jump market, in which the asset price of a firm follows a double exponential jump diffusion process, the value of the debt is driven by a geometric Brownian motion, and the default barrier follows a continuous stochastic process. Using the Gaver-Stehfest algorithm and the non-arbitrage asset pricing theory, we give the default probability of the first passage time, and more, derive the price of the Credit Default Swap.

A Survey of Multimodal Controllable Diffusion Models

Diffusion models have recently emerged as powerful generative models,producing high-fidelity samples across domains.Despite this,they have two key challenges,including improving the time-consuming iterative generation process and controlling and steering the generation process.Existing surveys provide broad overviews of diffusion model advancements.However,they lack comprehensive coverage specifically centered on techniques for controllable generation.This survey seeks to address this gap by providing a comprehensive and coherent review on controllable generation in diffusion models.We provide a detailed taxonomy defining controlled generation for diffusion models.Controllable generation is categorized based on the formulation,methodologies,and evaluation metrics.By enumerating the range of methods researchers have developed for enhanced control,we aim to establish controllable diffusion generation as a distinct subfield warranting dedicated focus.With this survey,we contextualize recent results,provide the dedicated treatment of controllable diffusion model generation,and outline limitations and future directions.To demonstrate applicability,we highlight controllable diffusion techniques for major computer vision tasks application.By consolidating methods and applications for controllable diffusion models,we hope to catalyze further innovations in reliable and scalable controllable generation.

Diffusion analysis and modeling of kinetic behavior for treatment of brine water using electrodialysis process

In this study,the removal of monovalent and divalent cations,Nat,Kt,Mg2t,and Ca2t,in a diluted solution from Chott-El Jerid Lake,Tunisia,was investigated with the electrodialysis technique.The process was tested using two cation-exchange membranes:sulfonated polyether sulfone cross-linked with 10%hexamethylenediamine(HEXCl)and sulfonated polyether sulfone grafted with octylamine(S-PESOS).The commercially available membrane Nafion®was used for comparison.The results showed that Nafion®and S-PESOS membranes had similar removal behaviors,and the investigated cations were ranked in the following descending order in terms of their demineralization rates:Nat>Ca2t>Mg2t>Kt.Divalent cations were more effectively removed by HEXCl than by monovalent cations.The plots based on the WebereMorris model showed a strong linearity.This reveals that intra-particle diffusion was not the removal rate-determining step,and the removal process was controlled by two or more concurrent mechanisms.The Boyd plots did not pass through their origin,and the sole controlling step was determined by film-diffusion resistance,especially after a long period of electrodialysis.Additionally,a semi-empirical model was established to simulate the temporal variation of the treatment process,and the physical significance and values of model parameters were compared for the three membranes.The findings of this study indicate that HEXCl and S-PESOS membranes can be efficiently utilized for water softening,especially when effluents are highly loaded with calcium and magnesium ions.

Construction of Clonal Growth Models in Trifolium repens

The suitability of models for describing the clonal growth of Trifolium repens population was discussed. The results showed that deterministic models were inadequate for describing its clonal growth, but the diffusion models and the randomwalk models suited for the clonal growth characteristics of the population. And it was found that random-walk models were better than diffusion models for describing a population in an environment with rich natural resources, and the latter was better in a poor environment.