Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

OptoGPT: A foundation model for inverse design in optical multilayer thin film structures

Optical multilayer thin film structures have been widely used in numerous photonic applications.However,existing inverse design methods have many drawbacks because they either fail to quickly adapt to different design targets,or are difficult to suit for different types of structures,e.g.,designing for different materials at each layer.These methods also cannot accommodate versatile design situations under different angles and polarizations.In addition,how to benefit practical fabrications and manufacturing has not been extensively considered yet.In this work,we introduce OptoGPT(Opto Generative Pretrained Transformer),a decoder-only transformer,to solve all these drawbacks and issues simultaneously.

Centrifuge modeling of a large-scale surcharge on adjacent foundation

This study investigates the ground and structural response of adjacent raft foundations induced by largescale surcharge by ore in soft soil areas through a 130g centrifuge modeling test with an innovative layered loading device.The prototype of the test is a coastal iron ore yard with a natural foundation of deep soft soil.Therefore,it is necessary to adopt some measures to reduce the influence of the large-scale surcharge on the adjacent raft foundation,such as installing stone columns for foundation treatment.Under an acceleration of 130 g,the model conducts similar simulations of iron ore,stone columns,and raft foundation structures.The tested soil mass has dimensions of 900 mm×700 mm×300 mm(lengthwidthdepth),which is remodeled from the soil extracted from the drilling holes.The test conditions are consistent with the actual engineering conditions and the effects of four-level loading conditions on the composite foundation of stone columns,unreinforced zone,and raft foundations are studied.An automatic layer-by-layer loading device was innovatively developed to simulate the loading process of actual engineering more realistically.The composite foundation of stone columns had a large settlement after the loading,forming an obvious settlement trough and causing the surface of the unreinforced zone to rise.The 12 m surcharge loading causes a horizontal displacement of 13.19 cm and a vertical settlement of 1.37 m in the raft foundation.The stone columns located on both sides of the unreinforced zone suffered significant shear damage at the sand-mud interface.Due to the reinforcement effect of stone columns,the sand layer below the top of the stone columns moves less.Meanwhile,the horizontal earth pressure in the raft foundation zone increases slowly.The stone columns will form new drainage channels and accelerate the dissipation of excess pore pressure.

Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

Performance of composite foundations with different load transfer platforms and substratum stiffness over silty clay

The semi-rigid pile-supported composite foundation is widely used in highway projects due to its effectiveness in increasing the bearing capacity and stability of foundations.It is crucial to understand the stress distribution across the embankment width and the behaviour of unreinforced foundations.Thus,five centrifuge tests were conducted to examine the bearing and deformation behaviours of NPRS(Non-Connected Piled Raft Systems)and GRPS(GeosyntheticReinforced Pile-Supported systems)with varying substratum stiffness,then a comparative analysis was conducted on embankment settlement,pressures underneath the embankments,and axial forces along the piles.The results indicated that greater substratum stiffness correlates with reduced settlement and deformation at various depths.Deformation occurring 5 meters from the embankment toe includes settlement in NPRS and upward movement in GRPS.The potential sliding surface is primarily located within the embankment in NPRS,whereas it may extend through both the embankment and foundation in GRPS.The pile-soil stress ratio and efficiency in NPRS are higher than in GRPS across the embankment.The axial force borne by end-bearing piles is significantly greater than that by floating piles.As the buried depth increases,the axial force in GRPS initially rises then declines,whereas in NPRS,it remains relatively constant within a certain range before decreasing.This study aids in assessing the applicability of composite foundations in complex railway environments and provides a reference for procedural measures under similar conditions.

Local Scour Mechanisms and Prediction Methods Around Offshore Wind Turbine Foundations:Insights and Future Directions

Local scour around offshore wind turbine foundations presents a considerable challenge due to its potential influence on structural stability,driven by hydrodynamic forces.While research has made strides in comprehending scouring mechanisms,notable complexities persist,specifically with newer foundation types.Addressing these limitations is vital for advancing our understanding of scour mechanisms and for improving mitigation strategies in offshore wind energy development.This review synthesizes current findings on local scour across various offshore foundations,encompassing field observations,data-driven approaches,turbulence-sediment interactions,scour evolution processes,influencing factors,and numerical model advancements.The objective is to enrich our understanding of local scour mechanisms.In addition,future research directions are outlined,including the development of robust arti-ficial intelligence models for accurate predictions,the exploration of vortex structure characteristics,and the refinement of numerical models to strengthen prediction capabilities while minimizing computational efforts.

Oscillating Spacetime: The Foundation of the Universe

In this article, spacetime is modeled as a quantum mechanical sonic medium consisting of Planck length oscillations at Planck frequency. Planck length-time oscillations give spacetime its physical constants of c, G and ħ. Oscillating spacetime is proposed to be the single universal field that generates and unifies everything in the universe. The 17 fields of quantum field theory are modeled as lower frequency resonances of oscillating spacetime. A model of an electron is proposed to be a rotating soliton wave in this medium. An electron appears to have wave-particle duality even though it is fundamentally a quantized wave. This soliton wave can momentarily be smaller than a proton in a high energy collision or can have a relatively large volume of an atom’s orbital wave function. Finding an electron causes it to undergo a superluminal collapse to a smaller wave size. This gives an electron its particle-like properties when detected. The proposed wave-based electron model is tested and shown to have an electron’s approximate energy, de Broglie wave properties and undetectable volume. Most important, this electron model is shown to also generate an electron’s electrostatic and gravitational forces. The gravitational properties are derived from the nonlinearity of this medium. When an electron’s gravitational and electrostatic forces are modeled as distortions of soliton waves, the equations become very simple, and a clear connection emerges between these forces. For example, the gravitational force between two Planck masses equals the electrostatic force between two Planck charges. Both force magnitudes equal ħc/r2.

Multi-modal Gesture Recognition using Integrated Model of Motion, Audio and Video

Gesture recognition is used in many practical applications such as human-robot interaction, medical rehabilitation and sign language. With increasing motion sensor development, multiple data sources have become available, which leads to the rise of multi-modal gesture recognition. Since our previous approach to gesture recognition depends on a unimodal system, it is difficult to classify similar motion patterns. In order to solve this problem, a novel approach which integrates motion, audio and video models is proposed by using dataset captured by Kinect. The proposed system can recognize observed gestures by using three models. Recognition results of three models are integrated by using the proposed framework and the output becomes the final result. The motion and audio models are learned by using Hidden Markov Model. Random Forest which is the video classifier is used to learn the video model. In the experiments to test the performances of the proposed system, the motion and audio models most suitable for gesture recognition are chosen by varying feature vectors and learning methods. Additionally, the unimodal and multi-modal models are compared with respect to recognition accuracy. All the experiments are conducted on dataset provided by the competition organizer of MMGRC, which is a workshop for Multi-Modal Gesture Recognition Challenge. The comparison results show that the multi-modal model composed of three models scores the highest recognition rate. This improvement of recognition accuracy means that the complementary relationship among three models improves the accuracy of gesture recognition. The proposed system provides the application technology to understand human actions of daily life more precisely.

Multi-Modal Military Event Extraction Based on Knowledge Fusion

Event extraction stands as a significant endeavor within the realm of information extraction,aspiring to automatically extract structured event information from vast volumes of unstructured text.Extracting event elements from multi-modal data remains a challenging task due to the presence of a large number of images and overlapping event elements in the data.Although researchers have proposed various methods to accomplish this task,most existing event extraction models cannot address these challenges because they are only applicable to text scenarios.To solve the above issues,this paper proposes a multi-modal event extraction method based on knowledge fusion.Specifically,for event-type recognition,we use a meticulous pipeline approach that integrates multiple pre-trained models.This approach enables a more comprehensive capture of the multidimensional event semantic features present in military texts,thereby enhancing the interconnectedness of information between trigger words and events.For event element extraction,we propose a method for constructing a priori templates that combine event types with corresponding trigger words.This approach facilitates the acquisition of fine-grained input samples containing event trigger words,thus enabling the model to understand the semantic relationships between elements in greater depth.Furthermore,a fusion method for spatial mapping of textual event elements and image elements is proposed to reduce the category number overload and effectively achieve multi-modal knowledge fusion.The experimental results based on the CCKS 2022 dataset show that our method has achieved competitive results,with a comprehensive evaluation value F1-score of 53.4%for the model.These results validate the effectiveness of our method in extracting event elements from multi-modal data.

Model test on vertical bearing capacity of X-section concrete pile raft foundation in silica sand

To reveal the bearing capacity of the X-section concrete piles pile raft foundation in silica sand,a series of vertical load tests are carried out.The X-section concrete piles are compared with circular section pile with the same section area.The load−settlement curves,axial force and skin friction,strain on concave and convex edge of the pile,pile-sand stress ratio,distributions of side and tip resistance are presented.The results show that bearing capacity of the X section concrete pile raft foundation is much larger than that of the circular pile raft foundation.Besides,compared with the circular pile,the peak axial force of X-section piles under raft is deeper and smaller while the neutral point of X-section concrete pile is deeper.Moreover,the strain on the concave edge is much larger than that on the convex edge of the pile,and the convex edge has more potential in bearing capacity as the vertical load increases.The X-section pile has higher pile-sand stress ratios and load-sharing between side resistance and tip resistance.Above all,the X-section concrete pile can significantly increase the bearing capacity of pile-raft foundations in silica sand.

Three-dimensional forward modeling and response characteristics analysis of foundation pit leakage electric-field considering electrokinetic effect

The traditional ground direct current method is not suitable for leakage detection of underground diaphragm walls in foundation pits because of its low accuracy and poor anti-noise ability.Here,we propose a joint surface-borehole observation device for leakage electric fi eld detection to achieve rapid measurement of the electric fi eld distribution characteristics at ground level in the foundation pit,thus enabling rapid localization of leakage points.We first establish the mechanism and basic equation of the leakage electric field response by combining the electric field formed by electrokinetic effect(EK)and the stable electric fi eld formed by conduction current in a combined leakage channel.Then,the fi nite–infi nite element coupling method is used to solve the electric fi eld equation to simulate the responses of a three-dimensional foundation pit leakage model.Furthermore,we conduct numerical simulations of diff erent pit models to investigate the infl uencing factors of the detection device and response characteristics of the change in the properties of the leakage channel.The results demonstrate that the proposed joint surface-borehole observation device can effi ciently reveal anomalous potential caused by leakage,and the amplitude of the electric fi eld generated by EK can eff ectively strengthen the leakage electric fi eld signal at the leakage,thus improving detection accuracy and effi ciency.

Numerical Analysis and Centrifuge Modeling of Shallow Foundations

The influence of non-coaxial constitutive model on predictions of dense sand behavior is investigated in this paper. The non-coaxial model with strain softening plasticity is applied into finite-element program ABAQUS, which is first used to predict the stress-strain behavior and the non-coaxial characteristic between the orientations of the principal stress and principal plastic strain rate in simple shear tests. The model is also used to predict load settlement responses and bearing capacity factors of shallow foundations. A series of centrifuge tests for shallow foundations on saturated dense sand are performed under drained conditions and the test results are compared with the corresponding numerical results. Various footing dimensions, depths of embedment, and footing shapes are considered in these tests. In view of the load settlement relationships, the stiffness of the load-displacement curves is significantly affected by the non-coaxial model compared with those predicted by the coaxial model, and a lower value of non-coaxial modulus gives a softer response. Considering the soil behavior at failure, the coaxial model predictions of bearing capacity factors are more advanced than those of centrifuge test results and the non-coaxial model results;besides, the non-coaxial model gives better predictions. The non-coaxial model predictions are closer to those of the centrifuge results when a proper non-coaxial plastic modulus is chosen.

A discrete model for response estimation of soil-structure systems with embedded foundations

The need for simplified physical models representing frequency dependent soil impedances has been the motivation behind many researches throughout history. Generally, such models are generated to capture impedance functions in a wide range of excitation frequencies, which leads to relatively complex models. That is while there is just a limited range of frequencies that really influence the response of the structure. Here, a new methodology based on the response-matching concept is proposed, which can lead to the development of simpler discrete models. The idea is then used to upgrade an existing simple model of surface foundations to the case of embedded foundations. The applicability of the model in both frequency domain and time domain analyses of soil-structure systems with embedded foundations is discussed. Moreover, the accuracy of the results is compared with another existing discrete model for embedded foundations.

Cone model for two surface foundations on layered soil

In this paper, the cone model is applied to the vibration analysis of two foundations on a layered soil half space. In the analysis, the total stress field in the subsoil is divided into the free-field and the scattering field. Seed＇s simplified method is adopted for the free-field analysis, while the cone model is proposed for analyzing the dynamic scattering stress wave field. The shear stress field and the compressive stress field in the layered stratum with two scattering sources are calculated by shear cone and compressive cone, respectively. Furthermore, the stress fields in the subsoil with two foundations are divided into six zones, and the P wave and S wave are analyzed in each zone. Numerical results are provided to illustrate features of the added stress field for two surface foundations under vertical and horizontal sinusoidal force excitation. The proposed cone model may be useful in handling some of the complex problems associated with multi-scattering sources.

Settlement Prediction for Buildings Surrounding Foundation Pits Based on a Stationary Auto-regression Model

To ensure the safety of buildings surrounding foundation pits, a study was made on a settlement monitoring and trend prediction method. A statistical testing method for analyzing the stability of a settlement monitoring datum has been discussed. According to a comprehensive survey, data of 16 stages at operating control point, were verified by a standard t test to determine the stability of the operating control point. A stationary auto-regression model, AR(p), used for the observation point settlement prediction has been investigated. Given the 16 stages of the settlement data at an observation point, the applicability of this model was analyzed. Settlement of last four stages was predicted using the stationary auto-regression model AR (1); the maximum difference between predicted and measured values was 0.6 mm, indicating good prediction results of the model. Hence, this model can be applied to settlement predictions for buildings surrounding foundation pits.

Analysis of prompt fission neutron spectrum and multiplicity for ^(237)Np(n, f) in the frame of multi-modal Los Alamos model

The improved version of Los Alamos model with the multi-modal fission approach is used to analyse the prompt fission neutron spectrum and multiplicity for the neutron-induced fission of 237Np. The spectra of neutrons emitted from fragments for the three most dominant fission modes （standard Ⅰ, standard Ⅱ and superlong） are calculated separately and the total spectrum is synthesized. The multi-modal parameters contained in the spectrum model are determined on the basis of experimental data of fission fragment mass distributions. The calculated total prompt fission neutron spectrum and multiplicity are better agreement with the experimental data than those obtained from the conventional treatment of the Los Alamos model.

Calculation of Prompt Fission Neutron from ^(233)U(n, f) Reaction byMulti-Modal Los Alamos Model

An attempt is made to improve the evaluation of the prompt fission neutron emis- sion from 233U（n, f） reaction for incident neutron energies below 6 MeV. The multi-modal fission approach is applied to the improved version of Los Alamos model and the point by point model. The prompt fission neutron spectra and the prompt fission neutron as a function of fragment mass （usually named ＂sawtooth＂ data） v（A） are calculated independently for the three most dominant fission modes （standard I, standard II and superlong）, and the total spectra and v（A） are syn- thesized. The multi-modal parameters are determined on the basis of experimental data of fission fragment mass distributions. The present calculation results can describe the experimental data very well, and the proposed treatment is thus a useful tool for prompt fission neutron emission prediction.

Coupling model for assessing anti-seepage behavior of curtain under dam foundation

The seepage under a dam foundation is mainly controlled by the performance of the curtain.Its anti-seepage behavior may be weakened by the long term physic-chemical actions from groundwater.According to seepage hydraulics and geochemistry theory,a coupling model for assessing the behavior of the curtain under a dam foundation is set up,which consists of seepage module,solute transport module,geochemistry module and curtain erosion module,solved by FEM.A case study was carried out.The result shows that the curtain efficiency is weakened all the time.Aqueous calcium from the curtain is always in dissolution during the stress period for simulation,which leads to the increasing amount in groundwater reaching 846.35-865.312 g/m3.Within the domain,reaction extent differs in different parts of the curtain.The dissolution of Ca(OH)2 accounts to 877.884 g/m3 near the bottom and is much higher than that of the other parts.The erosion is much more serious near the bottom of the curtain than the other parts,which is the same to the upstream and downstream.Calcium dissolution is mainly controlled by hydraulic condition and dispersion,and it varies in a non-linear way within the domain.

Centrifugal model test and numerical simulation of vertical earth pressure on soft foundation box culvert

To obtain the vertical earth pressure on a soft foundation box culvert and investigate the interaction of the soil-culvert-foundation system, both a centrifugal model test and a numerical simulation were conducted and the comparisons with the current methods to determine the load on a culvert were completed. The results of the model test and numerical analysis are in satisfactory agreement, which shows that the direction of the shear stress between the culvert and the adjacent embankment depends on the differential settlement between them. A vertical earth pressure concentration appears on the culvert with a rigid piles foundation because of a downward shear stress. The ratio of the load on a soft foundation culvert and the overburden pressure above the culvert raises first and then decreases as the backfill height increases. In order to reduce the load on a culvert, it is suggested to limit the stiffness difference of the foundations under the culvert and embankment and to use a light backfill over the culvert.

A Method for the Damage Detection of Pile Foundation in High-Pile Wharf Based on A Curvature Mode Deletion Model

As the top of the pile foundation in high-pile wharf is connected to the superstructure and most of the pile bodies are located below the water surface, traditional damage detection methods are greatly limited in their application to pile foundation in service. In the present study, a new method for pile foundation damage detection is developed based on the curve shape of the curvature mode difference(CMD) before and after damage. In the method, the influence at each node on the overall CMD curve shape is analyzed through a data deletion model, statistical characteristic indexes are established to reflect the difference between damaged and undamaged units, and structural damage is accurately detected. The effectiveness and robustness of the method are verified by a finite element model(FEM) of high-pile wharf under different damage conditions and different intensities of Gaussian white noise. The applicability of the method is then experimentally validated by a physical model of high-pile wharf. Both the FEM and the experimental results show that the method is capable of detecting pile foundation damage in noisy curvature mode and has strong application potential.