Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

Tree species classification using deep learning and RGB optical images obtained by an unmanned aerial vehicle

The diversity of tree species and the complexity of land use in cities create challenging issues for tree species classification.The combination of deep learning methods and RGB optical images obtained by unmanned aerial vehicles(UAVs) provides a new research direction for urban tree species classification.We proposed an RGB optical image dataset with 10 urban tree species,termed TCC10,which is a benchmark for tree canopy classification(TCC).TCC10 dataset contains two types of data:tree canopy images with simple backgrounds and those with complex backgrounds.The objective was to examine the possibility of using deep learning methods(AlexNet,VGG-16,and ResNet-50) for individual tree species classification.The results of convolutional neural networks(CNNs) were compared with those of K-nearest neighbor(KNN) and BP neural network.Our results demonstrated:(1) ResNet-50 achieved an overall accuracy(OA) of 92.6% and a kappa coefficient of 0.91 for tree species classification on TCC10 and outperformed AlexNet and VGG-16.(2) The classification accuracy of KNN and BP neural network was less than70%,while the accuracy of CNNs was relatively higher.(3)The classification accuracy of tree canopy images with complex backgrounds was lower than that for images with simple backgrounds.For the deciduous tree species in TCC10,the classification accuracy of ResNet-50 was higher in summer than that in autumn.Therefore,the deep learning is effective for urban tree species classification using RGB optical images.

基于新健康因子的锂电池健康状态估计和剩余寿命预测

容量和内阻是评估锂离子电池健康状态和预测其剩余寿命的重要指标,然而电池容量和内阻难以直接在线测量.通过分析锂离子电池充电过程中电流和电压的变化特征后提取出两种健康因子,并且证明所提因子与电池容量高度相关,进一步建立了用于锂电池容量估计的两因子线性回归模型.在此基础上,通过结合BP(back propagation)神经网络和粒子群优化思想设计锂离子电池健康状态估计算法.考虑到锂电池的健康状态和剩余使用寿命之间存在一定的映射关系,因此再利用所提取的健康因子和其健康状态估计结果设计了锂电池的剩余使用寿命预测算法.实验结果表明,所提取的健康因子能够准确地进行电池容量估计并应用于在线评估锂离子电池的健康状态和预测其剩余使用寿命.

Reliability analysis for WSN based on a modular k-out-of-n system

A hierarchical clustering model for a wireless sensor network (WSN) which takes into account the node fault and the balance of energy consumption is built. With the sensing and transmission as the main tasks of the WSN, its working mode is converted to a modular k-out-of-n system whose each modular is also a k-out-of-n structure. By using the theory of signature and its extending, the reliability of the WSN is computed. Finally, the reliability of WSN changing with the number of nodes, coverage requirements and node lifetimes is given by the numerical analysis. The proposed reliability analysis model is suitable for a large-scale WSN. © 2017 Beijing Institute of Aerospace Information.

Tracking Algorithm Based on Improved Interacting Multiple Model Particle Filter

Measurements are always interfered with glint noise in a radar target tracking system, which makes the performance of traditional filtering fall sharply and even divergent.Against this problem, a new Interactive Multiple Model Particle Filter (IMMPF) algorithm is proposed for target tracking by introducing PF into Interactive Multiple Model (IMM).Different from the general method to select importance density function from PF, the particles are extracted from observation likelihood function within depending on observation noises.Observation noise is modelled, and the latest observation is fused, then the target can be effectively tracked.Finally, the optimized method is simulated with respect to bearings-only tracking of maneuvering target in a glint noise environment.Compared with the existing filtering algorithms, it turns out that the developed filtering algorithm is more efficient and closer to the real-time tracking requirement of high maneuvering targets.

Game theory in network security for digital twins in industry

To ensure the safe operation of industrial digital twins network and avoid the harm to the system caused by hacker invasion,a series of discussions on network security issues are carried out based on game theory.From the perspective of the life cycle of network vulnerabilities,mining and repairing vulnerabilities are analyzed by applying evolutionary game theory.The evolution process of knowledge sharing among white hats under various conditions is simulated,and a game model of the vulnerability patch cooperative development strategy among manufacturers is constructed.On this basis,the differential evolution is introduced into the update mechanism of the Wolf Colony Algorithm(WCA)to produce better replacement individuals with greater probability from the perspective of both attack and defense.Through the simulation experiment,it is found that the convergence speed of the probability(X)of white Hat 1 choosing the knowledge sharing policy is related to the probability(x0)of white Hat 2 choosing the knowledge sharing policy initially,and the probability(y0)of white hat 2 choosing the knowledge sharing policy initially.When y0?0.9,X converges rapidly in a relatively short time.When y0 is constant and x0 is small,the probability curve of the“cooperative development”strategy converges to 0.It is concluded that the higher the trust among the white hat members in the temporary team,the stronger their willingness to share knowledge,which is conducive to the mining of loopholes in the system.The greater the probability of a hacker attacking the vulnerability before it is fully disclosed,the lower the willingness of manufacturers to choose the"cooperative development"of vulnerability patches.Applying the improved wolf colonyco-evolution algorithm can obtain the equilibrium solution of the"attack and defense game model",and allocate the security protection resources according to the importance of nodes.This study can provide an effective solution to protect the network security for digital twins in the industry.

Operation of Off-grid Power Supply System Using IoT Monitoring Platform for Oil and Gas Pipeline Based on RESOC

An oil and gas pipeline monitoring platform uses internet of things(IoT)to ensure safe operation in remote and unattended areas,through automatic monitoring and systematic control on equipment such as the cut-off valves and cathodic protection systems.The continuity and stability of power supplies for various equipment of an oil and gas pipeline IoT monitoring platform is crucial.There is no single universal off-grid power supply method that is optimal for an oil and gas pipeline IoT monitoring platform in all different contexts.Therefore,it is necessary to select a suitable one according to the specific geographical location and meteorological conditions.This paper proposes an off-grid power supply system comprised of a reversible solid oxide fuel cell(RESOC),photovoltaic(PV)and battery.Minimum operating costs and the reliability of system operations under constraint conditions are the key determining objectives.A“PV+battery+RESOC”system operational optimization model is established.Based on the model,three types of off-grid power supply schemes are proposed,and three geographical locations with different meteorological conditions are selected as practical application scenarios.The Matlab Cplex solver is used to solve the different power supply modes of the three regions.And finally,the power supply scheme with the best reliability and economy under different geographical environments and meteorological conditions is obtained.

An Imperfect Grouping Maintenance Strategy for Multi-Component Systems Using the Survival Signature

This paper presents an imperfect maintenance strategy for the multi-component systems.The proposed maintenance strategy takes into account two types of maintenance actions,namely preventive maintenance(PM)and corrective maintenance(CM).The imperfect effect of PM is modeled on the basis of the hybrid hazard rate model.Meanwhile,a new structure importance measure based on the survival signature is presented.Using this new importance measure method,an adjustment function is designed to update the PM maintenance threshold.For CM actions,a decision rule relying on the criticality level of components is introduced.In order to judge the criticality level of components,a novel structure updating method based on the survival signature is proposed.Moreover,a maintenance model considering economic dependence among components is developed.A 10-component system is finally introduced to illustrate the use and advantages of the proposed maintenance strategy.

Using GMOSTNet for Tree Detection Under Complex Illumination and Morphological Occlusion

Trees are an integral part of the forestry ecosystem.In forestry work,the precise acquisition of tree morphological parameters and attributes is affected by complex illumination and tree morphology.In order to minimize a series of inestimable problems,such as yield reduction,ecological damage,and destruction,caused by inaccurate acquisition of tree location information,this paper proposes a ground tree detection method GMOSTNet.Based on the four types of tree species in the GMOST dataset and Faster R-CNN,it extracted the features of the trees,generate candidate regions,classification,and other operations.By reducing the influence of illumination and occlusion factors during experimentation,more detailed information of the input image was obtained.Meanwhile,regarding false detections caused by inappropriate approximations,the deviation and proximity of the proposal were adjusted.The experimental results showed that the AP value of the four tree species is improved after using GMOSTNet,and the overall accuracy increases from the original 87.25%to 93.25%.