Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

OptoGPT: A foundation model for inverse design in optical multilayer thin film structures

Optical multilayer thin film structures have been widely used in numerous photonic applications.However,existing inverse design methods have many drawbacks because they either fail to quickly adapt to different design targets,or are difficult to suit for different types of structures,e.g.,designing for different materials at each layer.These methods also cannot accommodate versatile design situations under different angles and polarizations.In addition,how to benefit practical fabrications and manufacturing has not been extensively considered yet.In this work,we introduce OptoGPT(Opto Generative Pretrained Transformer),a decoder-only transformer,to solve all these drawbacks and issues simultaneously.

Centrifuge modeling of a large-scale surcharge on adjacent foundation

This study investigates the ground and structural response of adjacent raft foundations induced by largescale surcharge by ore in soft soil areas through a 130g centrifuge modeling test with an innovative layered loading device.The prototype of the test is a coastal iron ore yard with a natural foundation of deep soft soil.Therefore,it is necessary to adopt some measures to reduce the influence of the large-scale surcharge on the adjacent raft foundation,such as installing stone columns for foundation treatment.Under an acceleration of 130 g,the model conducts similar simulations of iron ore,stone columns,and raft foundation structures.The tested soil mass has dimensions of 900 mm×700 mm×300 mm(lengthwidthdepth),which is remodeled from the soil extracted from the drilling holes.The test conditions are consistent with the actual engineering conditions and the effects of four-level loading conditions on the composite foundation of stone columns,unreinforced zone,and raft foundations are studied.An automatic layer-by-layer loading device was innovatively developed to simulate the loading process of actual engineering more realistically.The composite foundation of stone columns had a large settlement after the loading,forming an obvious settlement trough and causing the surface of the unreinforced zone to rise.The 12 m surcharge loading causes a horizontal displacement of 13.19 cm and a vertical settlement of 1.37 m in the raft foundation.The stone columns located on both sides of the unreinforced zone suffered significant shear damage at the sand-mud interface.Due to the reinforcement effect of stone columns,the sand layer below the top of the stone columns moves less.Meanwhile,the horizontal earth pressure in the raft foundation zone increases slowly.The stone columns will form new drainage channels and accelerate the dissipation of excess pore pressure.

Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

Performance of composite foundations with different load transfer platforms and substratum stiffness over silty clay

The semi-rigid pile-supported composite foundation is widely used in highway projects due to its effectiveness in increasing the bearing capacity and stability of foundations.It is crucial to understand the stress distribution across the embankment width and the behaviour of unreinforced foundations.Thus,five centrifuge tests were conducted to examine the bearing and deformation behaviours of NPRS(Non-Connected Piled Raft Systems)and GRPS(GeosyntheticReinforced Pile-Supported systems)with varying substratum stiffness,then a comparative analysis was conducted on embankment settlement,pressures underneath the embankments,and axial forces along the piles.The results indicated that greater substratum stiffness correlates with reduced settlement and deformation at various depths.Deformation occurring 5 meters from the embankment toe includes settlement in NPRS and upward movement in GRPS.The potential sliding surface is primarily located within the embankment in NPRS,whereas it may extend through both the embankment and foundation in GRPS.The pile-soil stress ratio and efficiency in NPRS are higher than in GRPS across the embankment.The axial force borne by end-bearing piles is significantly greater than that by floating piles.As the buried depth increases,the axial force in GRPS initially rises then declines,whereas in NPRS,it remains relatively constant within a certain range before decreasing.This study aids in assessing the applicability of composite foundations in complex railway environments and provides a reference for procedural measures under similar conditions.

Local Scour Mechanisms and Prediction Methods Around Offshore Wind Turbine Foundations:Insights and Future Directions

Local scour around offshore wind turbine foundations presents a considerable challenge due to its potential influence on structural stability,driven by hydrodynamic forces.While research has made strides in comprehending scouring mechanisms,notable complexities persist,specifically with newer foundation types.Addressing these limitations is vital for advancing our understanding of scour mechanisms and for improving mitigation strategies in offshore wind energy development.This review synthesizes current findings on local scour across various offshore foundations,encompassing field observations,data-driven approaches,turbulence-sediment interactions,scour evolution processes,influencing factors,and numerical model advancements.The objective is to enrich our understanding of local scour mechanisms.In addition,future research directions are outlined,including the development of robust arti-ficial intelligence models for accurate predictions,the exploration of vortex structure characteristics,and the refinement of numerical models to strengthen prediction capabilities while minimizing computational efforts.

Oscillating Spacetime: The Foundation of the Universe

In this article, spacetime is modeled as a quantum mechanical sonic medium consisting of Planck length oscillations at Planck frequency. Planck length-time oscillations give spacetime its physical constants of c, G and ħ. Oscillating spacetime is proposed to be the single universal field that generates and unifies everything in the universe. The 17 fields of quantum field theory are modeled as lower frequency resonances of oscillating spacetime. A model of an electron is proposed to be a rotating soliton wave in this medium. An electron appears to have wave-particle duality even though it is fundamentally a quantized wave. This soliton wave can momentarily be smaller than a proton in a high energy collision or can have a relatively large volume of an atom’s orbital wave function. Finding an electron causes it to undergo a superluminal collapse to a smaller wave size. This gives an electron its particle-like properties when detected. The proposed wave-based electron model is tested and shown to have an electron’s approximate energy, de Broglie wave properties and undetectable volume. Most important, this electron model is shown to also generate an electron’s electrostatic and gravitational forces. The gravitational properties are derived from the nonlinearity of this medium. When an electron’s gravitational and electrostatic forces are modeled as distortions of soliton waves, the equations become very simple, and a clear connection emerges between these forces. For example, the gravitational force between two Planck masses equals the electrostatic force between two Planck charges. Both force magnitudes equal ħc/r2.

Stiffness and Shear Stress Distribution of Glulam Beams in Elastic-Plastic Stage:Theory,Experiments and Numerical Modelling

Traditional methods focus on the ultimate bending moment of glulam beams and the fracture failure of materials with defects,which usually depends on empirical parameters.There is no systematic theoretical method to predict the stiffness and shear distribution of glulam beams in elastic-plastic stage,and consequently,the failure of such glulam beams cannot be predicted effectively.To address these issues,an analytical method considering material nonlinearity was proposed for glulam beams,and the calculating equations of deflection and shear stress distribution for different failure modes were established.The proposed method was verified by experiments and numerical models under the corresponding conditions.Results showed that the theoretical calculations were in good agreement with experimental and numerical results,indicating that the equations proposed in this paper were reliable and accurate for such glulam beams with wood material in the elastic-plastic stage ignoring the influence of mechanic properties in radial and tangential directions of wood.Furthermore,the experimental results reported by the previous studies indicated that the method was applicable and could be used as a theoretical reference for predicting the failure of glulam beams.

Elastic-plastic analysis for pile-anchor supporting system of deep foundation pit

By using numerical analysis methods to simulate the deep excavation,a lot of analyses are established on the basis of two-dimensional plane strain,ignoring the fact that foundation pit possesses three dimensions. For soil constitutive relation,people always take linear and nonlinear model,without considering the plastic behavior of soil. Using plastic-elastic hardening model to simulate constitutive relation of soil characteristics,the authors carried out mechanical analysis for pit excavation and support. The results show that the analysis for the stress state of pile anchor system is an effective way which provides theoretical basis for calculation of soil displacement.

Model test on vertical bearing capacity of X-section concrete pile raft foundation in silica sand

To reveal the bearing capacity of the X-section concrete piles pile raft foundation in silica sand,a series of vertical load tests are carried out.The X-section concrete piles are compared with circular section pile with the same section area.The load−settlement curves,axial force and skin friction,strain on concave and convex edge of the pile,pile-sand stress ratio,distributions of side and tip resistance are presented.The results show that bearing capacity of the X section concrete pile raft foundation is much larger than that of the circular pile raft foundation.Besides,compared with the circular pile,the peak axial force of X-section piles under raft is deeper and smaller while the neutral point of X-section concrete pile is deeper.Moreover,the strain on the concave edge is much larger than that on the convex edge of the pile,and the convex edge has more potential in bearing capacity as the vertical load increases.The X-section pile has higher pile-sand stress ratios and load-sharing between side resistance and tip resistance.Above all,the X-section concrete pile can significantly increase the bearing capacity of pile-raft foundations in silica sand.

Three-dimensional forward modeling and response characteristics analysis of foundation pit leakage electric-field considering electrokinetic effect

The traditional ground direct current method is not suitable for leakage detection of underground diaphragm walls in foundation pits because of its low accuracy and poor anti-noise ability.Here,we propose a joint surface-borehole observation device for leakage electric fi eld detection to achieve rapid measurement of the electric fi eld distribution characteristics at ground level in the foundation pit,thus enabling rapid localization of leakage points.We first establish the mechanism and basic equation of the leakage electric field response by combining the electric field formed by electrokinetic effect(EK)and the stable electric fi eld formed by conduction current in a combined leakage channel.Then,the fi nite–infi nite element coupling method is used to solve the electric fi eld equation to simulate the responses of a three-dimensional foundation pit leakage model.Furthermore,we conduct numerical simulations of diff erent pit models to investigate the infl uencing factors of the detection device and response characteristics of the change in the properties of the leakage channel.The results demonstrate that the proposed joint surface-borehole observation device can effi ciently reveal anomalous potential caused by leakage,and the amplitude of the electric fi eld generated by EK can eff ectively strengthen the leakage electric fi eld signal at the leakage,thus improving detection accuracy and effi ciency.

Numerical Analysis and Centrifuge Modeling of Shallow Foundations

The influence of non-coaxial constitutive model on predictions of dense sand behavior is investigated in this paper. The non-coaxial model with strain softening plasticity is applied into finite-element program ABAQUS, which is first used to predict the stress-strain behavior and the non-coaxial characteristic between the orientations of the principal stress and principal plastic strain rate in simple shear tests. The model is also used to predict load settlement responses and bearing capacity factors of shallow foundations. A series of centrifuge tests for shallow foundations on saturated dense sand are performed under drained conditions and the test results are compared with the corresponding numerical results. Various footing dimensions, depths of embedment, and footing shapes are considered in these tests. In view of the load settlement relationships, the stiffness of the load-displacement curves is significantly affected by the non-coaxial model compared with those predicted by the coaxial model, and a lower value of non-coaxial modulus gives a softer response. Considering the soil behavior at failure, the coaxial model predictions of bearing capacity factors are more advanced than those of centrifuge test results and the non-coaxial model results;besides, the non-coaxial model gives better predictions. The non-coaxial model predictions are closer to those of the centrifuge results when a proper non-coaxial plastic modulus is chosen.

A discrete model for response estimation of soil-structure systems with embedded foundations

The need for simplified physical models representing frequency dependent soil impedances has been the motivation behind many researches throughout history. Generally, such models are generated to capture impedance functions in a wide range of excitation frequencies, which leads to relatively complex models. That is while there is just a limited range of frequencies that really influence the response of the structure. Here, a new methodology based on the response-matching concept is proposed, which can lead to the development of simpler discrete models. The idea is then used to upgrade an existing simple model of surface foundations to the case of embedded foundations. The applicability of the model in both frequency domain and time domain analyses of soil-structure systems with embedded foundations is discussed. Moreover, the accuracy of the results is compared with another existing discrete model for embedded foundations.

A Method for the Damage Detection of Pile Foundation in High-Pile Wharf Based on A Curvature Mode Deletion Model

As the top of the pile foundation in high-pile wharf is connected to the superstructure and most of the pile bodies are located below the water surface, traditional damage detection methods are greatly limited in their application to pile foundation in service. In the present study, a new method for pile foundation damage detection is developed based on the curve shape of the curvature mode difference(CMD) before and after damage. In the method, the influence at each node on the overall CMD curve shape is analyzed through a data deletion model, statistical characteristic indexes are established to reflect the difference between damaged and undamaged units, and structural damage is accurately detected. The effectiveness and robustness of the method are verified by a finite element model(FEM) of high-pile wharf under different damage conditions and different intensities of Gaussian white noise. The applicability of the method is then experimentally validated by a physical model of high-pile wharf. Both the FEM and the experimental results show that the method is capable of detecting pile foundation damage in noisy curvature mode and has strong application potential.

Cone model for two surface foundations on layered soil

In this paper, the cone model is applied to the vibration analysis of two foundations on a layered soil half space. In the analysis, the total stress field in the subsoil is divided into the free-field and the scattering field. Seed＇s simplified method is adopted for the free-field analysis, while the cone model is proposed for analyzing the dynamic scattering stress wave field. The shear stress field and the compressive stress field in the layered stratum with two scattering sources are calculated by shear cone and compressive cone, respectively. Furthermore, the stress fields in the subsoil with two foundations are divided into six zones, and the P wave and S wave are analyzed in each zone. Numerical results are provided to illustrate features of the added stress field for two surface foundations under vertical and horizontal sinusoidal force excitation. The proposed cone model may be useful in handling some of the complex problems associated with multi-scattering sources.

Settlement Prediction for Buildings Surrounding Foundation Pits Based on a Stationary Auto-regression Model

To ensure the safety of buildings surrounding foundation pits, a study was made on a settlement monitoring and trend prediction method. A statistical testing method for analyzing the stability of a settlement monitoring datum has been discussed. According to a comprehensive survey, data of 16 stages at operating control point, were verified by a standard t test to determine the stability of the operating control point. A stationary auto-regression model, AR(p), used for the observation point settlement prediction has been investigated. Given the 16 stages of the settlement data at an observation point, the applicability of this model was analyzed. Settlement of last four stages was predicted using the stationary auto-regression model AR (1); the maximum difference between predicted and measured values was 0.6 mm, indicating good prediction results of the model. Hence, this model can be applied to settlement predictions for buildings surrounding foundation pits.

Coupling model for assessing anti-seepage behavior of curtain under dam foundation

The seepage under a dam foundation is mainly controlled by the performance of the curtain.Its anti-seepage behavior may be weakened by the long term physic-chemical actions from groundwater.According to seepage hydraulics and geochemistry theory,a coupling model for assessing the behavior of the curtain under a dam foundation is set up,which consists of seepage module,solute transport module,geochemistry module and curtain erosion module,solved by FEM.A case study was carried out.The result shows that the curtain efficiency is weakened all the time.Aqueous calcium from the curtain is always in dissolution during the stress period for simulation,which leads to the increasing amount in groundwater reaching 846.35-865.312 g/m3.Within the domain,reaction extent differs in different parts of the curtain.The dissolution of Ca(OH)2 accounts to 877.884 g/m3 near the bottom and is much higher than that of the other parts.The erosion is much more serious near the bottom of the curtain than the other parts,which is the same to the upstream and downstream.Calcium dissolution is mainly controlled by hydraulic condition and dispersion,and it varies in a non-linear way within the domain.

Centrifugal model test and numerical simulation of vertical earth pressure on soft foundation box culvert

To obtain the vertical earth pressure on a soft foundation box culvert and investigate the interaction of the soil-culvert-foundation system, both a centrifugal model test and a numerical simulation were conducted and the comparisons with the current methods to determine the load on a culvert were completed. The results of the model test and numerical analysis are in satisfactory agreement, which shows that the direction of the shear stress between the culvert and the adjacent embankment depends on the differential settlement between them. A vertical earth pressure concentration appears on the culvert with a rigid piles foundation because of a downward shear stress. The ratio of the load on a soft foundation culvert and the overburden pressure above the culvert raises first and then decreases as the backfill height increases. In order to reduce the load on a culvert, it is suggested to limit the stiffness difference of the foundations under the culvert and embankment and to use a light backfill over the culvert.

Research status and application of artificial intelligence large models in the oil and gas industry

This article elucidates the concept of large model technology,summarizes the research status of large model technology both domestically and internationally,provides an overview of the application status of large models in vertical industries,outlines the challenges and issues confronted in applying large models in the oil and gas sector,and offers prospects for the application of large models in the oil and gas industry.The existing large models can be briefly divided into three categories:large language models,visual large models,and multimodal large models.The application of large models in the oil and gas industry is still in its infancy.Based on open-source large language models,some oil and gas enterprises have released large language model products using methods like fine-tuning and retrieval augmented generation.Scholars have attempted to develop scenario-specific models for oil and gas operations by using visual/multimodal foundation models.A few researchers have constructed pre-trained foundation models for seismic data processing and interpretation,as well as core analysis.The application of large models in the oil and gas industry faces challenges such as current data quantity and quality being difficult to support the training of large models,high research and development costs,and poor algorithm autonomy and control.The application of large models should be guided by the needs of oil and gas business,taking the application of large models as an opportunity to improve data lifecycle management,enhance data governance capabilities,promote the construction of computing power,strengthen the construction of“artificial intelligence+energy”composite teams,and boost the autonomy and control of large model technology.

Study on Unloading Creep Characteristics of the Soil and Application of the Stress-Dependent Creep Model in Suction Caisson Foundation

As the anchoring foundation of the tension leg platform(TLP),suction caisson foundation is subjected to the long-term vertical pullout loads.But there are few studies on the mechanism of the unloading creep of soft clay and long-term uplift bearing capacity of suction caisson foundations.To address this problem,unloading creep tests of soft clay were carried out to analyze the strain development with time under different confining pressures.The test results show that the creep curve rapidly develops in the early stage and tends to stabilize in the later stage.The unloading deviator stress is higher,the unloading creep deformation is greater and the soft clay has typical nonlinear creep characteristics.Therefore,by introducing the creep model and considering the influence of the deviator stress,the stress-dependent Merchant model is proposed to describe the unloading creep of soft clay.Then,the stress-dependent Merchant model is extended to a three-dimension constitutive model,and a finite element subroutine is developed to establish a finite element analysis method for analyzing the long-term uplift capacity of suction caisson foundations and validated with the long-term uplift bearing capacity results of caisson model.