A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

A multilayer network diffusion-based model for reviewer recommendation

With the rapid growth of manuscript submissions,finding eligible reviewers for every submission has become a heavy task.Recommender systems are powerful tools developed in computer science and information science to deal with this problem.However,most existing approaches resort to text mining techniques to match manuscripts with potential reviewers,which require high-quality textual information to perform well.In this paper,we propose a reviewer recommendation algorithm based on a network diffusion process on a scholar-paper multilayer network,with no requirement for textual information.The network incorporates the relationship of scholar-paper pairs,the collaboration among scholars,and the bibliographic coupling among papers.Experimental results show that our proposed algorithm outperforms other state-of-the-art recommendation methods that use graph random walk and matrix factorization and methods that use machine learning and natural language processing,with improvements of over 7.62%in recall,5.66%in hit rate,and 47.53%in ranking score.Our work sheds light on the effectiveness of multilayer network diffusion-based methods in the reviewer recommendation problem,which will help to facilitate the peer-review process and promote information retrieval research in other practical scenes.

Research on Restoration of Murals Based on Diffusion Model and Transformer

Due to the limitations of a priori knowledge and convolution operation,the existing image restoration techniques cannot be directly applied to the cultural relics mural restoration,in order to more accurately restore the original appearance of the cultural relics mural images,an image restoration based on the denoising diffusion probability model(Denoising Diffusion Probability Model(DDPM))and the Transformer method.The process involves two steps:in the first step,the damaged mural image is firstly utilized as the condition to generate the noise image,using the time,condition and noise image patch as the inputs to the noise prediction network,capturing the global dependencies in the input sequence through the multi-attentionmechanismof the input sequence and feedforward neural network processing,and designing a long skip connection between the shallow and deep layers in the Transformer blocks between the shallow and deep layers using long skip connections to fuse the feature information of global and local outputs to maintain the overall consistency of the restoration results;In the second step,taking the noisy image as a condition to direct the diffusion model to back sample to generate the restored image.Experiment results show that the PSNR and SSIM of the proposedmethod are improved by 2%to 9%and 1%to 3.3%,respectively,which are compared to the comparison methods.This study proposed synthesizes the advantages of the diffusionmodel and deep learningmodel to make themural restoration results more accurate.

Dynamics modeling and optimal control for multi-information diffusion in Social Internet of Things

As an ingenious convergence between the Internet of Things and social networks,the Social Internet of Things(SIoT)can provide effective and intelligent information services and has become one of the main platforms for people to spread and share information.Nevertheless,SIoT is characterized by high openness and autonomy,multiple kinds of information can spread rapidly,freely and cooperatively in SIoT,which makes it challenging to accurately reveal the characteristics of the information diffusion process and effectively control its diffusion.To this end,with the aim of exploring multi-information cooperative diffusion processes in SIoT,we first develop a dynamics model for multi-information cooperative diffusion based on the system dynamics theory in this paper.Subsequently,the characteristics and laws of the dynamical evolution process of multi-information cooperative diffusion are theoretically investigated,and the diffusion trend is predicted.On this basis,to further control the multi-information cooperative diffusion process efficiently,we propose two control strategies for information diffusion with control objectives,develop an optimal control system for the multi-information cooperative diffusion process,and propose the corresponding optimal control method.The optimal solution distribution of the control strategy satisfying the control system constraints and the control budget constraints is solved using the optimal control theory.Finally,extensive simulation experiments based on real dataset from Twitter validate the correctness and effectiveness of the proposed model,strategy and method.

2-D Modeling and Calculations of Stratospheric Ozone and Influences of Convection, Diffusion, and Time

An engineering system approach of 2-D cylindrical model of transient mass balance calculations of ozone and other concerned chemicals along with fourteen photolysis, ozone-generating and ozone-depleting chemical reaction equations was developed, validated, and used for studying the ozone concentrations, distribution and peak of the layer, ozone depletion and total ozone abundance in the stratosphere. The calculated ozone concentrations and profile at both the Equator and a 60˚N location were found to follow closely with the measured data. The calculated average ozone concentration was within 1% of the measured average, and the deviation of ozone profiles was within 14%. The monthly evolution of stratospheric ozone concentrations and distribution above the Equator was studied with results discussed in details. The influences of slow air movement in both altitudinal and radial directions on ozone concentrations and profile in the stratosphere were explored and discussed. Parametric studies of the influences of gas diffusivities of ozone DO3 and active atomic oxygen DO on ozone concentrations and distributions were also studied and delineated. Having both influences through physical diffusion and chemical reactions, the diffusivity (and diffusion) of atomic oxygen DO was found to be more sensitive and important than that of ozone DO3 on ozone concentrations and distribution. The 2-D ozone model present in this paper for stratospheric ozone and its layer and depletion is shown to be robust, convenient, efficient, and executable for analyzing the complex ozone phenomena in the stratosphere. .

BeFOI: A Novel Method Based on Conditional Diffusion Model for Medical Image Denoising

The progress in medical imaging technology highlights the importance of image quality for effective diagnosis and treatment.Yet,noise during capture and transmission can compromise image accuracy and reliability,complicating clinical decisions.The rising interest in diffusion models has led to their exploration of denoising images.We present Be-FOI(Better Fluoro Images),a weakly supervised model that uses cine images to denoise fluoroscopic images,both DR types.Trained through precise noise estimation and simulation,BeFOI employs Markov chains to denoise using only the fluoroscopic image as guidance.Our tests show that BeFOI outperforms other methods,reducing noise and enhancing clar-ity and diagnostic utility,making it an effective post-processing tool for medical images.

Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

Analytical evaluation of steady-state solute distribution in through- diffusion and membrane behavior test under non-perfectly flushing boundary conditions

The through-diffusion and membrane behavior testing procedure using a closed-system apparatus has been widely used for concurrent measurement of diffusion and membrane efficiency coefficients of low-permeability clay-based barrier materials.However,the common assumption of perfectly flushing conditions at the specimen boundaries could induce errors in analyses of the diffusion coefficients and membrane efficiencies.In this study,an innovative pseudo three-dimensional(3D)analytical method was proposed to evaluate solute distribution along the boundary surfaces of the soil-porous disks system,considering the non-perfectly flushing conditions.The results were consistent with numerical models under two scenarios considering different inflow/outflow positions.The proposed model has been demonstrated to be an accurate and reliable method to estimate solute distributions along the bound-aries.The calculated membrane efficiency coefficient and diffusion coefficient based on the proposed analytical method are more accurate,resulting in up to 50%less relative error than the traditional approach that adopts the arithmetic mean value of the influent and effluent concentrations.The retar-dation factor of the clay specimen also can be calculated with a revised cumulative mass approach.Finally,the simulated transient solute transport matched with experimental data from a multi-stage through-diffusion and membrane behavior test,validating the accuracy of the proposed method.

Unveiling the Re,Cr,and I diffusion in saturated compacted bentonite using machine-learning methods

The safety assessment of high-level radioactive waste repositories requires a high predictive accuracy for radionuclide diffusion and a comprehensive understanding of the diffusion mechanism.In this study,a through-diffusion method and six machine-learning methods were employed to investigate the diffusion of ReO_(4)^(−),HCrO_(4)^(−),and I−in saturated compacted bentonite under different salinities and compacted dry densities.The machine-learning models were trained using two datasets.One dataset contained six input features and 293 instances obtained from the diffusion database system of the Japan Atomic Energy Agency(JAEA-DDB)and 15 publications.The other dataset,comprising 15,000 pseudo-instances,was produced using a multi-porosity model and contained eight input features.The results indicate that the former dataset yielded a higher predictive accuracy than the latter.Light gradient-boosting exhibited a higher prediction accuracy(R2=0.92)and lower error(MSE=0.01)than the other machine-learning algorithms.In addition,Shapley Additive Explanations,Feature Importance,and Partial Dependence Plot analysis results indicate that the rock capacity factor and compacted dry density had the two most significant effects on predicting the effective diffusion coefficient,thereby offering valuable insights.

LoRA模型微调Stable Diffusion的设计师风格服装生成方法

为控制Stable Diffusion模型准确生成特定设计师风格的服装图像,应用LoRA模型微调Stable Diffusion生成过程的方法实现风格化服装图像生成。以中国十佳时装服装设计师孙林的设计风格为例,首先对设计师的发布会服装进行系统分析,提取设计师的风格特征因子,根据设计师风格因子分别训练3个风格化LoRA模型,分别验证每个风格因子的表达效果。最后,通过3个LoRA模型叠加使用的方式,生成具有设计师风格特征的服装设计图像。结果表明:同时调用多个风格化LoRA模型微调Stable Diffusion的生成过程,能够使生成的服装符合设计师的服装风格,实现设计师风格的迁移设计,提高设计师进行服装系列设计和产品开发的效率。

Sulfate diffusion in coal pillar:experimental data and prediction model

The stability of coal pillar dams is crucial for the long-term service of underground reservoirs storing water or heat.Chemi-cal damage of coal dams induced by ions-atttacking in coal is one of the main reasons for the premature failure of coal dams.However,the diffusion process of harmful ions in coal is far from clear,limiting the reliability and durability of coal dam designs.This paper investigates sulfate diffusion in coal pillar through experimental and analytical methods.Coal specimens are prepared and exposed to sulfate solutions with different concentrations.The sulfate concentrations at different locations and time are measured.Based on experimental data and Fick's law,the time-dependent surface concentration of sulfate and diffusion coefficient are determined and formulated.Further,an analytical model for predicting sulfate diffusion in coal pillar is developed by considering dual time-dependent characteristics and Laplace transformations.Through comparisons with experimental data,the accuracy of the analytical model for predicting sulfate diffusion is verified.Further,sulfate diffusions in coal dams for different concentrations of sulfate in mine water are investigated.It has been found that the sulfate concen-tration of exposure surface and diffusion coefficient in coal are both time-dependent and increase with time.Conventional Fick's law is not able to predict the sulfate diffusion in coal pillar due to the dual time-dependent characteristics.The sulfate attacking makes the coal dam a typical heterogeneous gradient structure.For sulfate concentrations 0.01-0.20 mol/L in mine water,it takes almost 1.5 and 4 years for sulfate ions to diffuse 9.46 and 18.92 m,respectively.The experimental data and developed model provide a practical method for predicting sulfate diffusion in coal pillar,which helps the service life design of coal dams.

Molecular Dynamics, Physical Properties, Diffusion Coefficients and Activation Energy of the Lithium Oxide (Li-O) and Sodium Oxide (Na-O) Electrolyte (Cathode)

This work is a simulation model with the LAMMPS calculation code of an electrode based on alkali metal oxides (lithium, sodium and potassium) using the Lennard Jones potential. For a multiplicity of 8*8*8, we studied a gap-free model using molecular dynamics. Physical quantities such as volume and pressure of the Na-O and Li-O systems exhibit similar behaviors around the thermodynamic ensembles NPT and NVE. However, for the Na2O system, at a minimum temperature value, we observe a range of total energy values;in contrast, for the Li2O system, a minimum energy corresponds to a range of temperatures. Finally, for physicochemical properties, we studied the diffusion coefficient and activation energy of lithium and potassium oxides around their melting temperatures. The order of magnitude of the diffusion coefficients is given by the relation Dli-O >DNa-O for the multiplicity 8*8*8, while for the activation energy, the order is well reversed EaNa-O > EaLi-O.

Identification of the Diffusion Coefficients of Pollutants ( N O 3 - and Mn2+) through the Walls of Concrete Tank and Vulnerability of the Stored Water Quality

The populations of urban centers in Congo-Brazzaville have decided to develop various methods of water storage (concrete or masonry underground tanks) for domestic use, due to shortages in the distribution of water through the public network. However, questions remain as to the physico-chemical quality of the water stored in these tanks, when these structures are built in wet and relatively polluted areas. This paper presents a model of pollutant diffusion through the cementitious matrix (concrete) of tank walls simulated at a buried reservoir. The results of the experimental and numerical simulations show that certain concrete parameters, such as porosity, permeability and diffusivity, have a significant influence on the transfer of pollutants through the concrete walls, thus altering the physico-chemical quality of the stored water. The numerical models (1D) used to predict pollutant transfer and the quality of the stored water are consistent with those of the optimal control for identifying the diffusion coefficient. Major ion concentrations appear to be correlated with system porosity and diffusion coefficient. Nevertheless, the identification of the diffusion coefficient from the optimal control method, based on an explicit numerical resolution of a finite volume PDE for the approximation of the experiment, is not consistent with that of the optimal control method.

Determination of Natural Logarithm of Diffusion Coefficient and Activation Energy of Thin Layer Drying Process of Ginger Rhizome Slices

This study is an extension of the previous work done with ARS-680 Environmental Chamber. Drying is a complex operation that demands much energy and time. Drying is essentially important for preservation of ginger rhizome. Drying of ginger was modeled, and then the effective diffusion coefficient and activation energy were determined. For this purpose, the experiments were done at six levels of varied temperatures: 10°C, 20°C, 30°C, 40°C, 50°C and 60°C. The values of effective diffusion coefficients obtained in this work for the variously treated ginger rhizomes closely agreed with the average effective diffusion coefficients of other notable authors who determined the drying kinetics and convective heat transfer coefficients of ginger slices.

Numerical Simulation of Diffusion Type Traffic Flow Model Using Second-Order Lax-Wendroff Scheme Based on Exponential Velocity Density Function

In order to control traffic congestion, many mathematical models have been used for several decades. In this paper, we study diffusion-type traffic flow model based on exponential velocity density relation, which provides a non-linear second-order parabolic partial differential equation. The analytical solution of the diffusion-type traffic flow model is very complicated to approximate the initial density of the Cauchy problem as a function of x from given data and it may cause a huge error. For the complexity of the analytical solution, the numerical solution is performed by implementing an explicit upwind, explicitly centered, and second-order Lax-Wendroff scheme for the numerical solution. From the comparison of relative error among these three schemes, it is observed that Lax-Wendroff scheme gives less error than the explicit upwind and explicit centered difference scheme. The numerical, analytical analysis and comparative result discussion bring out the fact that the Lax-Wendroff scheme with exponential velocity-density relation of diffusion type traffic flow model is suitable for the congested area and shows a better fit in traffic-congested regions.

Study on the Mathematical Model of Technology Diffusion in Technical Economic System

In the technical and economic system,there is a certain interaction between various technologies,mainly in the form of technology diffusion.By constructing the mathematical model of technology diffusion,the trend of technology diffusion can be predicted,the reasonable distribution of science and technology productivity can be realized,and the policy optimization measures can be formulated on the basis of evaluating the existing policies.This paper first points out the basic idea of digital model of technology diffusion,and then focuses on the analysis of various parameters of mathematical model of technology diffusion,and puts forward the application conclusion of mathematical model of technology diffusion,in order to provide reference for relevant industry personnel.

DIFFUSION MODELLING IN GARNET FROM TSO MORARI ECLOGITE AND IMPLICATIONS FOR EXHUMATION MODELS

The results of recent geothermobarometric and geochronological investigations of scarce eclogites of the NW Himalaya (Tso Morari (Ladakh), India and Kaghan Valley, Pakistan) have caused a major rethink of tectonometamorphic models for India\|Asia collision. Numerous petrologic studies have been undertaken on the age and origin of metamorphism in the Higher Himalayan Crystallines (HHC) and Lesser Himalaya formations (LH) and their relationship to granite magmatism and movements along the Main Central Thrust (MCT) and South Tibetan Detachment Fault (STDF). However, all of these events are essentially Miocene (or younger) in age and can clearly be distinguished from subduction and exhumation processes undergone by the eclogites which are of Eocene age (Tonarini et al. 1993; Spencer & Gebauer; 1996; de Sigoyer et al. 1999) and relate to the very early stages of the collision. Eclogites of eastern Ladakh are mafic lenses found in granitic gneisses (Ordovician intrusive age: Girard & Bussy 1999) and their surrounding late Pre\|Cambrian to early Cambrian sedimentary units in the Tso Morari dome (see Steck et al. 1998). Detailed petrological and geochronological studies (Guillot et al. 1997; de Sigoyer et al. 1997, 1999) have identified an eclogite facies stage (2000±300)MPa, (580±60)℃ followed by isothermal decompression associated with glaucophane growth at around (1100±200)MPa. Dating of different phases by different methods yielded ages around 55Ma for this stage ((55±17) Ma, U\|Pb, Aln; (55±12) Ma, Lu\|Hf, Grt\|Cpx\|Rt; (55±7) Ma, Sm\|Nd, Grt\|Gln\|Rt). A subsequent amphibolite facies overprint at slightly higher temperature (610±70)℃ was dated at 45～48Ma (metabasite: (47±11) Ma, Sm\|Nd, Grt\|Hbl; metapelite: (45±4) Ma, Rb\|Sr, Mu\|Ap\|WR and (48±2) Ma, Ar\|Ar, Phe). By (30±1) Ma (Ar\|Ar, Bt\|Mu) retrogression into the greenschist facies had occurred (de Sigoyer et al. 1999). These data indicate a two stage history with early exhumation being much faster (>4mm/a) than the later evolution (1～2mm/a).

Effects of rotation and gravity on an electro-magneto-thermoelastic medium with diffusion and voids by using the Lord-Shulman and dual-phase-lag models

The effects of rotation and gravity on an electro-magneto-thermoelastic medium with diffusion and voids in a generalized thermoplastic half-space are studied by using the Lord-Shulman (L-S) model and the dual-phase-lag (DPL) model. The analytical solutions for the displacements, stresses, temperature, diffusion concentration, and volume fraction field with different values of the magnetic field, the rotation, the gravity, and the initial stress are obtained and portrayed graphically. The results indicate that the effects of gravity, rotation, voids, diffusion, initial stress, and electromagnetic field are very pronounced on the physical properties of the material.

基于Diffusion-Model的PM_(2.5)检测中数据样本扩充研究

在PM_(2.5)检测领域,生成对抗网络(GAN)是最为常见的数据集扩充方案,通常在GAN的训练中通过扩大生成器和鉴别器分布的支持度以防止鉴别器过拟合。然而,因难以找到合适的噪声分布,采用这种方法难以达到预期的数据集扩充目标。而结合Diffusion-Model框架构建的Diffusion-GAN数据生成模型可以缓解丢失问题来稳定训练梯度,并通过创建同一样本的不同噪声版本来增强数据,从而达到提高生成样本质量和生成器的多样性的目的。实验表明,Diffusion-GAN相较于基线算法,其训练性能和生成样本的保真度分别提高了15.1%和32.7%,能够满足PM_(2.5)空气质量检测数据扩充的需求。

Dynamic models of technology management capability development based on knowledge diffusion

Technology management is recognized as a key for organizations to achieve competitiveness. How to promote an organization’s technology management capability is of great significance in creating efficiencies and achieving a competitive edge. The knowledge essence of technology management capability is introduced and then the correlation between knowledge diffusion and the development of technology management capability is discussed. Further, the basic and extended dynamic models of the development of technology management capability are constructed, and is applied into an enterprise. The results show that the dynamic models can well explain how the knowledge improves the development of technology management capability, and they can be used as an useful tool by an enterprise to promote technology management capability. Finally, the managerial implications of the models are discussed.