Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

Smart industrial IoT empowered crowd sensing for safety monitoring in coal mine

The crowd sensing technology can realize the sensing and computing of people,machines,and environment in smart industrial IoT-based coal mine,which provides a solution for safety monitoring through distributed intelligence optimization.However,due to the difficulty of neural network training to achieve global optimality and the fact that traditional LSTM methods do not consider the relationship between adjacent machines,the accuracy of human body position prediction and pressure value prediction is not high.To solve these problems,this paper proposes a smart industrial IoT empowered crowd sensing for safety monitoring in coal mine.First,we propose a Particle Swarm Optimization-Elman Neural Network(PE)algorithm for the mobile human position prediction.Second,we propose an ADI-LSTM neural network prediction algorithm for pressure values of machines supports in underground mines.Among them,our proposed PE algorithm has the lowest average cumulative prediction error,and the trajectory fit rate is improved by 24.1%,13.9%and 8.7%compared with Kalman filtering,Elman and Kalman plus Elman algorithms,respectively.Meanwhile,compared with single-input ARIMA,RNN,LSTM,and GRU,the RMSE values of our proposed ADI-LSTM are reduced by 36.6%,52%,32%,and 13.7%,respectively;and the MAPE values are reduced by 0.0003%,0.9482%,1.1844%,and 0.3620%,respectively.

Challenge-based collaborative intrusion detection in software-defined networking: An evaluation

Software-Defined Networking(SDN)is an emerging architecture that enables a computer network to be intelligently and centrally controlled via software applications.It can help manage the whole network environment in a consistent and holistic way,without the need of understanding the underlying network structure.At present,SDN may face many challenges like insider attacks,i.e.,the centralized control plane would be attacked by malicious underlying devices and switches.To protect the security of SDN,effective detection approaches are indispensable.In the literature,challenge-based collaborative intrusion detection networks(CIDNs)are an effective detection framework in identifying malicious nodes.It calculates the nodes'reputation and detects a malicious node by sending out a special message called a challenge.In this work,we devise a challenge-based CIDN in SDN and measure its performance against malicious internal nodes.Our results demonstrate that such a mechanism can be effective in SDN environments.

3D Reconstruction for Motion Blurred Images Using Deep Learning-Based Intelligent Systems

The 3D reconstruction using deep learning-based intelligent systems can provide great help for measuring an individual’s height and shape quickly and accurately through 2D motion-blurred images.Generally,during the acquisition of images in real-time,motion blur,caused by camera shaking or human motion,appears.Deep learning-based intelligent control applied in vision can help us solve the problem.To this end,we propose a 3D reconstruction method for motion-blurred images using deep learning.First,we develop a BF-WGAN algorithm that combines the bilateral filtering(BF)denoising theory with a Wasserstein generative adversarial network(WGAN)to remove motion blur.The bilateral filter denoising algorithm is used to remove the noise and to retain the details of the blurred image.Then,the blurred image and the corresponding sharp image are input into the WGAN.This algorithm distinguishes the motion-blurred image from the corresponding sharp image according to the WGAN loss and perceptual loss functions.Next,we use the deblurred images generated by the BFWGAN algorithm for 3D reconstruction.We propose a threshold optimization random sample consensus(TO-RANSAC)algorithm that can remove the wrong relationship between two views in the 3D reconstructed model relatively accurately.Compared with the traditional RANSAC algorithm,the TO-RANSAC algorithm can adjust the threshold adaptively,which improves the accuracy of the 3D reconstruction results.The experimental results show that our BF-WGAN algorithm has a better deblurring effect and higher efficiency than do other representative algorithms.In addition,the TO-RANSAC algorithm yields a calculation accuracy considerably higher than that of the traditional RANSAC algorithm.

A blockchain-empowered AAA scheme in the large-scale HetNet

A Large-Scale Heterogeneous Network(LS-HetNet)integrates different networks into one uniform network system to provide seamless one-world network coverage.In LS-HetNet,various devices use different technologies to access heterogeneous networks and generate a large amount of data.For dealing with a large number of access requirements,these data are usually stored in the HetNet Domain Management Server(HDMS)of the current domain,and HDMS uses a centralized Authentication/Authorization/Auditing(AAA)scheme to protect the data.However,this centralized method easily causes the data to be modified or disclosed.To address this issue,we propose a blockchain-empowered AAA scheme for accessing data of LS-HetNet.Firstly,the account address of the blockchain is used as the identity authentication,and the access control permission of data is redesigned and stored on the blockchain,then processes of AAA are redefined.Finally,the experimental model on Ethereum private chain is built,and the results show that the scheme is not only secure but also decentral,without tampering and trustworthiness.