A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

A multilayer network diffusion-based model for reviewer recommendation

With the rapid growth of manuscript submissions,finding eligible reviewers for every submission has become a heavy task.Recommender systems are powerful tools developed in computer science and information science to deal with this problem.However,most existing approaches resort to text mining techniques to match manuscripts with potential reviewers,which require high-quality textual information to perform well.In this paper,we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network,with no requirement for textual information.The network incorporates the relationship of scholar-paper pairs,the collaboration among scholars,and the bibliographic coupling among papers.Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing,with improvements of over 7.62%in recall,5.66%in hit rate,and 47.53%in ranking score.Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem,which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

Research on Restoration of Murals Based on Diffusion Model and Transformer

Due to the limitations of a priori knowledge and convolution operation,the existing image restoration techniques cannot be directly applied to the cultural relics mural restoration,in order to more accurately restore the original appearance of the cultural relics mural images,an image restoration based on the denoising diffusion probability model(Denoising Diffusion Probability Model(DDPM))and the Transformer method.The process involves two steps:in the first step,the damaged mural image is firstly utilized as the condition to generate the noise image,using the time,condition and noise image patch as the inputs to the noise prediction network,capturing the global dependencies in the input sequence through the multi-attentionmechanismof the input sequence and feedforward neural network processing,and designing a long skip connection between the shallow and deep layers in the Transformer blocks between the shallow and deep layers using long skip connections to fuse the feature information of global and local outputs to maintain the overall consistency of the restoration results;In the second step,taking the noisy image as a condition to direct the diffusion model to back sample to generate the restored image.Experiment results show that the PSNR and SSIM of the proposedmethod are improved by 2%to 9%and 1%to 3.3%,respectively,which are compared to the comparison methods.This study proposed synthesizes the advantages of the diffusionmodel and deep learningmodel to make themural restoration results more accurate.

SeisResoDiff: Seismic resolution enhancement based on a diffusion model

High resolution of post-stack seismic data assists in better interpretation of subsurface structures as well as high accuracy of impedance inversion. Therefore, geophysicists consistently strive to acquire higher resolution seismic images in petroleum exploration. Although there have been successful applications of conventional signal processing and machine learning for post-stack seismic resolution enhancement,there is limited reference to the seismic applications of the recent emergence and rapid development of generative artificial intelligence. Hence, we propose to apply diffusion models, among the most popular generative models, to enhance seismic resolution. Specifically, we apply the classic diffusion model—denoising diffusion probabilistic model(DDPM), conditioned on the seismic data in low resolution, to reconstruct corresponding high-resolution images. Herein the entire scheme is referred to as SeisResoDiff. To provide a comprehensive and clear understanding of SeisResoDiff, we introduce the basic theories of diffusion models and detail the optimization objective's derivation with the aid of diagrams and algorithms. For implementation, we first propose a practical workflow to acquire abundant training data based on the generated pseudo-wells. Subsequently, we apply the trained model to both synthetic and field datasets, evaluating the results in three aspects: the appearance of seismic sections and slices in the time domain, frequency spectra, and comparisons with the synthetic data using real well-logging data at the well locations. The results demonstrate not only effective seismic resolution enhancement,but also additional denoising by the diffusion model. Experimental comparisons indicate that training the model on noisy data, which are more realistic, outperforms training on clean data. The proposed scheme demonstrates superiority over some conventional methods in high-resolution reconstruction and denoising ability, yielding more competitive results compared to our previous research.

Dynamics modeling and optimal control for multi-information diffusion in Social Internet of Things

As an ingenious convergence between the Internet of Things and social networks,the Social Internet of Things(SIoT)can provide effective and intelligent information services and has become one of the main platforms for people to spread and share information.Nevertheless,SIoT is characterized by high openness and autonomy,multiple kinds of information can spread rapidly,freely and cooperatively in SIoT,which makes it challenging to accurately reveal the characteristics of the information diffusion process and effectively control its diffusion.To this end,with the aim of exploring multi-information cooperative diffusion processes in SIoT,we first develop a dynamics model for multi-information cooperative diffusion based on the system dynamics theory in this paper.Subsequently,the characteristics and laws of the dynamical evolution process of multi-information cooperative diffusion are theoretically investigated,and the diffusion trend is predicted.On this basis,to further control the multi-information cooperative diffusion process efficiently,we propose two control strategies for information diffusion with control objectives,develop an optimal control system for the multi-information cooperative diffusion process,and propose the corresponding optimal control method.The optimal solution distribution of the control strategy satisfying the control system constraints and the control budget constraints is solved using the optimal control theory.Finally,extensive simulation experiments based on real dataset from Twitter validate the correctness and effectiveness of the proposed model,strategy and method.

Spatial patterns of the Brusselator model with asymmetric Lévy diffusion

The formation of spatial patterns is an important issue in reaction–diffusion systems.Previous studies have mainly focused on the spatial patterns in reaction–diffusion models equipped with symmetric diffusion(such as normal or fractional Laplace diffusion),namely,assuming that spatial environments of the systems are homogeneous.However,the complexity and heterogeneity of spatial environments of biochemical reactions in vivo can lead to asymmetric diffusion of reactants.Naturally,there arises an open question of how the asymmetric diffusion affects dynamical behaviors of biochemical reaction systems.To answer this,we build a general asymmetric L´evy diffusion model based on the theory of a continuous time random walk.In addition,we investigate the two-species Brusselator model with asymmetric L´evy diffusion,and obtain a general condition for the formation of Turing and wave patterns.More interestingly,we find that even though the Brusselator model with symmetric diffusion cannot produce steady spatial patterns for some parameters,the asymmetry of L´evy diffusion for this model can produce wave patterns.This is different from the previous result that wave instability requires at least a three-species model.In addition,the asymmetry of L´evy diffusion can significantly affect the amplitude and frequency of the spatial patterns.Our results enrich our knowledge of the mechanisms of pattern formation.

2-D Modeling and Calculations of Stratospheric Ozone and Influences of Convection, Diffusion, and Time

An engineering system approach of 2-D cylindrical model of transient mass balance calculations of ozone and other concerned chemicals along with fourteen photolysis, ozone-generating and ozone-depleting chemical reaction equations was developed, validated, and used for studying the ozone concentrations, distribution and peak of the layer, ozone depletion and total ozone abundance in the stratosphere. The calculated ozone concentrations and profile at both the Equator and a 60˚N location were found to follow closely with the measured data. The calculated average ozone concentration was within 1% of the measured average, and the deviation of ozone profiles was within 14%. The monthly evolution of stratospheric ozone concentrations and distribution above the Equator was studied with results discussed in details. The influences of slow air movement in both altitudinal and radial directions on ozone concentrations and profile in the stratosphere were explored and discussed. Parametric studies of the influences of gas diffusivities of ozone DO3 and active atomic oxygen DO on ozone concentrations and distributions were also studied and delineated. Having both influences through physical diffusion and chemical reactions, the diffusivity (and diffusion) of atomic oxygen DO was found to be more sensitive and important than that of ozone DO3 on ozone concentrations and distribution. The 2-D ozone model present in this paper for stratospheric ozone and its layer and depletion is shown to be robust, convenient, efficient, and executable for analyzing the complex ozone phenomena in the stratosphere. .

ON A MULTI-DELAY LOTKA-VOLTERRA PREDATOR-PREY MODEL WITH FEEDBACK CONTROLS AND PREY DIFFUSION

This article is focusing on a class of multi-delay predator-prey model with feedback controls and prey diffusion. By developing some new analysis methods and using the theory of differential inequalities as well as constructing a suitable Lyapunov function, we establish a set of easily verifiable sufficient conditions which guarantee the permanence of the system and the globally attractivity of positive solution for the predator-prey system.Furthermore, some conditions for the existence, uniqueness and stability of positive periodic solution for the corresponding periodic system are obtained by using the fixed point theory and some new analysis techniques. In additional, some numerical solutions of the equations describing the system are given to verify the obtained criteria are new, general, and easily verifiable. Finally, we still solve numerically the corresponding stochastic predator-prey models with multiplicative noise sources, and obtain some new interesting dynamical behaviors of the system.

BeFOI: A Novel Method Based on Conditional Diffusion Model for Medical Image Denoising

The progress in medical imaging technology highlights the importance of image quality for effective diagnosis and treatment.Yet,noise during capture and transmission can compromise image accuracy and reliability,complicating clinical decisions.The rising interest in diffusion models has led to their exploration of denoising images.We present Be-FOI(Better Fluoro Images),a weakly supervised model that uses cine images to denoise fluoroscopic images,both DR types.Trained through precise noise estimation and simulation,BeFOI employs Markov chains to denoise using only the fluoroscopic image as guidance.Our tests show that BeFOI outperforms other methods,reducing noise and enhancing clar-ity and diagnostic utility,making it an effective post-processing tool for medical images.

Note on a diffusive ratio-dependent predator-prey model

Subject to the homogeneous Neumann boundary condition, a ratio-dependent predator-prey reaction diffusion model is discussed. An improved result for the model is derived, that is, the unique positive constant steady state is the global stability. This is done using the comparison principle and establishing iteration schemes involving positive solutions supremum and infimum. The result indicates that the two species will ultimately distribute homogeneously in space. In fact, the comparison argument and iteration technique to be used in this paper can be applied to some other models. This method deals with the not-existence of a non-constant positive steady state for some reaction diffusion systems, which is rather simple but sufficiently effective.

Dynamic vaccine distribution model based on epidemic diffusion rule and clustering approach

Due to the fact that the emergency medicine distribution is vital to the quick response to urgent demand when an epidemic occurs, the optimal vaccine distribution approach is explored according to the epidemic diffusion rule and different urgency degrees of affected areas with the background of the epidemic outbreak in a given region. First, the SIQR （susceptible, infected, quarantined,recovered） epidemic model with pulse vaccination is introduced to describe the epidemic diffusion rule and obtain the demanded vaccine in each pulse. Based on the SIQR model, the affected areas are clustered by using the self-organizing map （SOM） neutral network to qualify the results. Then, a dynamic vaccine distribution model is formulated, incorporating the results of clustering the affected areas with the goals of both reducing the transportation cost and decreasing the unsatisfied demand for the emergency logistics network. Numerical study with twenty affected areas and four distribution centers is carried out. The corresponding numerical results indicate that the proposed approach can make an outstanding contribution to controlling the affected areas with a relatively high degree of urgency, and the comparison results prove that the performance of the clustering method is superior to that of the non-clustering method on controlling epidemic diffusion.

Constitutive model for methane desorption and diffusion based on pore structure differences between soft and hard coal

This paper aims to improve the accuracy and applicability of gas diffusion mathematical models from coal particles. Firstly, a new constitutive model for gas diffusion from coal particles with tri-disperse pore structure is constructed by considering the difference in characteristics between soft coal and hard coal.The analytical solution is then derived, that is, the quantitative relationship between gas diffusion rate(Qt/Q_∞) and diffusion time(t), The pore structure parameters of soft coal and hard coal from Juji coal mine are determined. Gas diffusion rules are numerically calculated and investigated by physical simulation methods. Lastly, the applicability of this model is verified. The results show that the homogeneous model only applies to the gas diffusion process of hard coal during the initial 10 min. The calculation results from this model and the physical experimental results of soft coal and hard coal are nearly identical during the initial 30 min.

A Three-Dimensional Model of Transport and Diffusion of Seeding Agents within Stratus

It is essential to learn the temporal and spatial concentration distributions and variations of seeding agents in cloud seeding of precipitation enhancement. A three-dimensional puff trajectory model incorporating a mesoscale nonhydrostatic model has been formulated, and is applied to simulating the transporting and diffusive characteristics of multiple line sources of seeding agents within super-cooled stratus. Several important factors are taken into consideration that affect the diffusion of seeding materials such as effects of topography and vertical wind shear, temporal and spatial variation of seeding parameters and wet deposition. The particles of seeding agents are assumed to be almost inert, they have no interaction with the particles of the cloud or precipitation except that they are washed out by precipitation. The model validity is demonstrated by the analyses and comparisons of model results, and checked by the sensitivity experiments of diffusive coefficients and atmospheric stratification. The advantage of this model includes not only its exact reflection of heterogeneity and unsteadiness of background fields, but also its good simulation of transport and diffusion of multiple line sources. The horizontal diffusion rate and the horizontal transport distance have been proposed that they usually were difficult to obtain in other models. In this simulation the horizontal diffusion rate is 0.82 m s(-1) for average of one hour, and the horizontal average transport distance reaches 65 km after 1 4 which are closely related to the background Fields.

A novel diffusion model considering curvature radius at the bonding interface in a titanium/steel explosive clad plate

This article introduces an element diffusion behavior model for a titanium/steel explosive clad plate characterized by a typical curved interface during the heat-treatment process. A series of heat-treatment experiments were conducted in the temperature range from 750℃ to 950℃, and the effects of heat-treatment parameters on the microstructural evolution and diffusion behavior were investigated by optical microscopy, scanning electron microscopy, X-ray diffraction analysis, and electron-probe microanalysis. Carbon atoms within the steel matrix were observed to diffuse toward the titanium matrix and to aggregate at the bonding interface at 850℃ or lower; in contrast, when the temperature exceeded 850℃, the mutual diffusion of Ti and Fe occurred, along with the diffusion of C atoms, resulting in the for- marion of Ti-Fe intermetallics （Fe2Ti/FeTi）. The diffusion distances of C, Ti, and Fe atoms increased with increasing heating temperature and/or holding time. On the basis of this diffusion behavior, a novel diffusion model was proposed. This model considers the effects of various factors, including the curvature radius of the curved interface, the diffusion coefficient, the heating temperature, and the holding rime. The experimental results show good agreement with the calculated values. The proposed model could clearly provide a general prediction of the elements＇ diffusion at both straight and curved interfaces.

OPTIMAL HARVESTING POLICY FOR INSHORE-OFFSHORE FISHERY MODEL WITH IMPULSIVE DIFFUSION

This article studies the inshore-offshore fishery model with impulsive diffusion. The existence and global asymptotic stability of both the trivial periodic solution and the positive periodic solution are obtained. The complexity of this system is also analyzed. Moreover, the optimal harvesting policy are given for the inshore subpopulation, which includes the maximum sustainable yield and the corresponding harvesting effort.

DIFFUSION MODELLING IN GARNET FROM TSO MORARI ECLOGITE AND IMPLICATIONS FOR EXHUMATION MODELS

The results of recent geothermobarometric and geochronological investigations of scarce eclogites of the NW Himalaya (Tso Morari (Ladakh), India and Kaghan Valley, Pakistan) have caused a major rethink of tectonometamorphic models for India\|Asia collision. Numerous petrologic studies have been undertaken on the age and origin of metamorphism in the Higher Himalayan Crystallines (HHC) and Lesser Himalaya formations (LH) and their relationship to granite magmatism and movements along the Main Central Thrust (MCT) and South Tibetan Detachment Fault (STDF). However, all of these events are essentially Miocene (or younger) in age and can clearly be distinguished from subduction and exhumation processes undergone by the eclogites which are of Eocene age (Tonarini et al. 1993; Spencer & Gebauer; 1996; de Sigoyer et al. 1999) and relate to the very early stages of the collision. Eclogites of eastern Ladakh are mafic lenses found in granitic gneisses (Ordovician intrusive age: Girard & Bussy 1999) and their surrounding late Pre\|Cambrian to early Cambrian sedimentary units in the Tso Morari dome (see Steck et al. 1998). Detailed petrological and geochronological studies (Guillot et al. 1997; de Sigoyer et al. 1997, 1999) have identified an eclogite facies stage (2000±300)MPa, (580±60)℃ followed by isothermal decompression associated with glaucophane growth at around (1100±200)MPa. Dating of different phases by different methods yielded ages around 55Ma for this stage ((55±17) Ma, U\|Pb, Aln; (55±12) Ma, Lu\|Hf, Grt\|Cpx\|Rt; (55±7) Ma, Sm\|Nd, Grt\|Gln\|Rt). A subsequent amphibolite facies overprint at slightly higher temperature (610±70)℃ was dated at 45～48Ma (metabasite: (47±11) Ma, Sm\|Nd, Grt\|Hbl; metapelite: (45±4) Ma, Rb\|Sr, Mu\|Ap\|WR and (48±2) Ma, Ar\|Ar, Phe). By (30±1) Ma (Ar\|Ar, Bt\|Mu) retrogression into the greenschist facies had occurred (de Sigoyer et al. 1999). These data indicate a two stage history with early exhumation being much faster (>4mm/a) than the later evolution (1～2mm/a).

Meso-scale modeling of chloride diffusion in concrete with consideration of effects of time and temperature

A meso-scale truss network model was developed to predict chloride diffusion in concrete. The model regards concrete as a three-phase composite of mortar matrix, coarse aggregates, and the interfacial transition zone （ITZ） between the mortar matrix and the aggregates. The diffusion coefficient of chloride in the mortar and the ITZ can be analytically determined with only the water-to-cement ratio and volume fraction of fine aggregates. Fick＇s second law of diffusion was used as the governing equation for chloride diffusion in a homogenous medium （e.g., mortar）; it was discretized and applied to the truss network model. The solution procedure of the truss network model based on the diffusion law and the meso-scale composite structure of concrete is outlined. Additionally, the dependence of the diffusion coefficient of chloride in the mortar and the ITZ on exposure duration and temperature is taken into account to illustrate their effect on chloride diffusion coefficient. The numerical results show that the exposure duration and environmental temperature play important roles in the diffusion rate of chloride ions in concrete. It is also concluded that the meso-scale truss network model can be applied to chloride transport analysis of damaged （or cracked） concrete.

STABILITY OF INNOVATION DIFFUSION MODEL WITH NONLINEAR ACCEPTANCE

In this article, an innovation diffusion model with the nonlinear acceptance is proposed to describe the dynamics of three competing products in a market. It is proved that the model admits a unique positive equilibrium, which is globally stable by excluding the existence of periodic solutions and by using the theory of three dimensional competition systems.

Effects of rotation and gravity on an electro-magneto-thermoelastic medium with diffusion and voids by using the Lord-Shulman and dual-phase-lag models

The effects of rotation and gravity on an electro-magneto-thermoelastic medium with diffusion and voids in a generalized thermoplastic half-space are studied by using the Lord-Shulman (L-S) model and the dual-phase-lag (DPL) model. The analytical solutions for the displacements, stresses, temperature, diffusion concentration, and volume fraction field with different values of the magnetic field, the rotation, the gravity, and the initial stress are obtained and portrayed graphically. The results indicate that the effects of gravity, rotation, voids, diffusion, initial stress, and electromagnetic field are very pronounced on the physical properties of the material.

Nonlinear predator-prey singularly perturbed Robin Problems for reaction diffusion systems

The nonlinear predator-prey reaction diffusion systems for singularly perturbed Robin Problems are considered. Under suitable conditions, the theory of differential inequalities can be used to study the asymptotic behavior of the solution for initial boundary value problems.