Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

3D Road Network Modeling and Road Structure Recognition in Internet of Vehicles

Internet of Vehicles (IoV) is a new system that enables individual vehicles to connect with nearby vehicles,people, transportation infrastructure, and networks, thereby realizing amore intelligent and efficient transportationsystem. The movement of vehicles and the three-dimensional (3D) nature of the road network cause the topologicalstructure of IoV to have the high space and time complexity.Network modeling and structure recognition for 3Droads can benefit the description of topological changes for IoV. This paper proposes a 3Dgeneral roadmodel basedon discrete points of roads obtained from GIS. First, the constraints imposed by 3D roads on moving vehicles areanalyzed. Then the effects of road curvature radius (Ra), longitudinal slope (Slo), and length (Len) on speed andacceleration are studied. Finally, a general 3D road network model based on road section features is established.This paper also presents intersection and road section recognition methods based on the structural features ofthe 3D road network model and the road features. Real GIS data from a specific region of Beijing is adopted tocreate the simulation scenario, and the simulation results validate the general 3D road network model and therecognitionmethod. Therefore, thiswork makes contributions to the field of intelligent transportation by providinga comprehensive approach tomodeling the 3Droad network and its topological changes in achieving efficient trafficflowand improved road safety.

Node Sociability Based Intelligent Routing for Post-Disaster Emergency Networks

In a post-disaster environment characterized by frequent interruptions in communication links,traditional wireless communication networks are ineffective.Although the“store-carry-forward”mechanism characteristic of Delay Tolerant Networks(DTNs)can transmit data from Internet of things devices to more reliable base stations or data centres,it also suffers from inefficient data transmission and excessive transmission delays.To address these challenges,we propose an intelligent routing strategy based on node sociability for post-disaster emergency network scenarios.First,we introduce an intelligent routing strategy based on node intimacy,which selects more suitable relay nodes and assigns the corresponding number of message copies based on comprehensive utility values.Second,we present an intelligent routing strategy based on geographical location of nodes to forward message replicas secondarily based on transmission utility values.Finally,experiments demonstrate the effectiveness of our proposed algorithm in terms of message delivery rate,network cost ratio and average transmission delay.

A Pricing-Based Cooperative Relay Selection Scheme for Reliable Communications

With the rapid development and application of energy harvesting technology,it has become a prominent research area due to its significant benefits in terms of green environmental protection,convenience,and high safety and efficiency.However,the uneven energy collection and consumption among IoT devices at varying distances may lead to resource imbalance within energy harvesting networks,thereby resulting in low energy transmission efficiency.To enhance the energy transmission efficiency of IoT devices in energy harvesting,this paper focuses on the utilization of collaborative communication,along with pricing-based incentive mechanisms and auction strategies.We propose a dynamic relay selection scheme,including a ladder pricing mechanism based on energy level and a Kuhn-Munkre Algorithm based on an auction theory employing a negotiation mechanism,to encourage more IoT devices to participate in the collaboration process.Simulation results demonstrate that the proposed algorithm outperforms traditional algorithms in terms of improving the energy efficiency of the system.

Fine-Grained Multivariate Time Series Anomaly Detection in IoT

Sensors produce a large amount of multivariate time series data to record the states of Internet of Things(IoT)systems.Multivariate time series timestamp anomaly detection(TSAD)can identify timestamps of attacks and malfunctions.However,it is necessary to determine which sensor or indicator is abnormal to facilitate a more detailed diagnosis,a process referred to as fine-grained anomaly detection(FGAD).Although further FGAD can be extended based on TSAD methods,existing works do not provide a quantitative evaluation,and the performance is unknown.Therefore,to tackle the FGAD problem,this paper first verifies that the TSAD methods achieve low performance when applied to the FGAD task directly because of the excessive fusion of features and the ignoring of the relationship’s dynamic changes between indicators.Accordingly,this paper proposes a mul-tivariate time series fine-grained anomaly detection(MFGAD)framework.To avoid excessive fusion of features,MFGAD constructs two sub-models to independently identify the abnormal timestamp and abnormal indicator instead of a single model and then combines the two kinds of abnormal results to detect the fine-grained anomaly.Based on this framework,an algorithm based on Graph Attention Neural Network(GAT)and Attention Convolutional Long-Short Term Memory(A-ConvLSTM)is proposed,in which GAT learns temporal features of multiple indicators to detect abnormal timestamps and A-ConvLSTM captures the dynamic relationship between indicators to identify abnormal indicators.Extensive simulations on a real-world dataset demonstrate that the proposed algorithm can achieve a higher F1 score and hit rate than the extension of existing TSAD methods with the benefit of two independent sub-models for timestamp and indicator detection.

Deep Learning-Based Robust Morphed Face Authentication Framework for Online Systems

The amalgamation of artificial intelligence(AI)with various areas has been in the picture for the past few years.AI has enhanced the functioning of several services,such as accomplishing better budgets,automating multiple tasks,and data-driven decision-making.Conducting hassle-free polling has been one of them.However,at the onset of the coronavirus in 2020,almost all worldly affairs occurred online,and many sectors switched to digital mode.This allows attackers to find security loopholes in digital systems and exploit them for their lucrative business.This paper proposes a three-layered deep learning(DL)-based authentication framework to develop a secure online polling system.It provides a novel way to overcome security breaches during the face identity(ID)recognition and verification process for online polling systems.This verification is done by training a pixel-2-pixel Pix2pix generative adversarial network(GAN)for face image reconstruction to remove facial objects present(if any).Furthermore,image-to-image matching is done by implementing the Siamese network and comparing the result of various metrics executed on feature embeddings to obtain the outcome,thus checking the electorate credentials.

Deep Learning-Based Model for Detection of Brinjal Weed in the Era of Precision Agriculture

The overgrowth of weeds growing along with the primary crop in the fields reduces crop production.Conventional solutions like hand weeding are labor-intensive,costly,and time-consuming;farmers have used herbicides.The application of herbicide is effective but causes environmental and health concerns.Hence,Precision Agriculture(PA)suggests the variable spraying of herbicides so that herbicide chemicals do not affect the primary plants.Motivated by the gap above,we proposed a Deep Learning(DL)based model for detecting Eggplant(Brinjal)weed in this paper.The key objective of this study is to detect plant and non-plant(weed)parts from crop images.With the help of object detection,the precise location of weeds from images can be achieved.The dataset is collected manually from a private farm in Gandhinagar,Gujarat,India.The combined approach of classification and object detection is applied in the proposed model.The Convolutional Neural Network(CNN)model is used to classify weed and non-weed images;further DL models are applied for object detection.We have compared DL models based on accuracy,memory usage,and Intersection over Union(IoU).ResNet-18,YOLOv3,CenterNet,and Faster RCNN are used in the proposed work.CenterNet outperforms all other models in terms of accuracy,i.e.,88%.Compared to other models,YOLOv3 is the least memory-intensive,utilizing 4.78 GB to evaluate the data.

Early Diagnosis of Lung Tumors for Extending Patients’ Life Using Deep Neural Networks

The medical community has more concern on lung cancer analysis.Medical experts’physical segmentation of lung cancers is time-consuming and needs to be automated.The research study’s objective is to diagnose lung tumors at an early stage to extend the life of humans using deep learning techniques.Computer-Aided Diagnostic(CAD)system aids in the diagnosis and shortens the time necessary to detect the tumor detected.The application of Deep Neural Networks(DNN)has also been exhibited as an excellent and effective method in classification and segmentation tasks.This research aims to separate lung cancers from images of Magnetic Resonance Imaging(MRI)with threshold segmentation.The Honey hook process categorizes lung cancer based on characteristics retrieved using several classifiers.Considering this principle,the work presents a solution for image compression utilizing a Deep Wave Auto-Encoder(DWAE).The combination of the two approaches significantly reduces the overall size of the feature set required for any future classification process performed using DNN.The proposed DWAE-DNN image classifier is applied to a lung imaging dataset with Radial Basis Function(RBF)classifier.The study reported promising results with an accuracy of 97.34%,whereas using the Decision Tree(DT)classifier has an accuracy of 94.24%.The proposed approach(DWAE-DNN)is found to classify the images with an accuracy of 98.67%,either as malignant or normal patients.In contrast to the accuracy requirements,the work also uses the benchmark standards like specificity,sensitivity,and precision to evaluate the efficiency of the network.It is found from an investigation that the DT classifier provides the maximum performance in the DWAE-DNN depending on the network’s performance on image testing,as shown by the data acquired by the categorizers themselves.

A Novel Secure Scan Design Based on Delayed Physical Unclonable Function

The advanced integrated circuits have been widely used in various situations including the Internet of Things,wireless communication,etc.But its manufacturing process exists unreliability,so cryptographic chips must be rigorously tested.Due to scan testing provides high test coverage,it is applied to the testing of cryptographic integrated circuits.However,while providing good controllability and observability,it also provides attackers with a backdoor to steal keys.In the text,a novel protection scheme is put forward to resist scan-based attacks,in which we first use the responses generated by a strong physical unclonable function circuit to solidify fuseantifuse structures in a non-linear shift register(NLSR),then determine the scan input code according to the configuration of the fuse-antifuse structures and the styles of connection between the NLSR cells and the scan cells.If the key is right,the chip can be tested normally;otherwise,the data in the scan chain cannot be propagated normally,it is also impossible for illegal users to derive the desired scan data.The proposed technique not only enhances the security of cryptographic chips,but also incurs acceptable overhead.

Secure and Efficient Data Transmission Scheme Based on Physical Mechanism

Many Internet of things application scenarios have the characteristics of limited hardware resources and limited energy supply,which are not suitable for traditional security technology.The security technology based on the physicalmechanism has attracted extensive attention.How to improve the key generation rate has always been one of the urgent problems to be solved in the security technology based on the physical mechanism.In this paper,superlattice technology is introduced to the security field of Internet of things,and a high-speed symmetric key generation scheme based on superlattice for Internet of things is proposed.In order to ensure the efficiency and privacy of data transmission,we also combine the superlattice symmetric key and compressive sensing technology to build a lightweight data transmission scheme that supports data compression and data encryption at the same time.Theoretical analysis and experimental evaluation results show that the proposed scheme is superior to the most closely related work.

Block Verification Mechanism Based on Zero-Knowledge Proof in Blockchain

Since transactions in blockchain are based on public ledger verification,this raises security concerns about privacy protection.And it will cause the accumulation of data on the chain and resulting in the low efficiency of block verification,when the whole transaction on the chain is verified.In order to improve the efficiency and privacy protection of block data verification,this paper proposes an efficient block verification mechanism with privacy protection based on zeroknowledge proof(ZKP),which not only protects the privacy of users but also improves the speed of data block verification.There is no need to put the whole transaction on the chain when verifying block data.It just needs to generate the ZKP and root hash with the transaction information,then save them to the smart contract for verification.Moreover,the ZKP verification in smart contract is carried out to realize the privacy protection of the transaction and efficient verification of the block.When the data is validated,the buffer accepts the complete transaction,updates the transaction status in the cloud database,and packages up the chain.So,the ZKP strengthens the privacy protection ability of blockchain,and the smart contracts save the time cost of block verification.

DoS Attack Detection Based on Deep Factorization Machine in SDN

Software-Defined Network(SDN)decouples the control plane of network devices from the data plane.While alleviating the problems presented in traditional network architectures,it also brings potential security risks,particularly network Denial-of-Service(DoS)attacks.While many research efforts have been devoted to identifying new features for DoS attack detection,detection methods are less accurate in detecting DoS attacks against client hosts due to the high stealth of such attacks.To solve this problem,a new method of DoS attack detection based on Deep Factorization Machine(DeepFM)is proposed in SDN.Firstly,we select the Growth Rate of Max Matched Packets(GRMMP)in SDN as detection feature.Then,the DeepFM algorithm is used to extract features from flow rules and classify them into dense and discrete features to detect DoS attacks.After training,the model can be used to infer whether SDN is under DoS attacks,and a DeepFM-based detection method for DoS attacks against client host is implemented.Simulation results show that our method can effectively detect DoS attacks in SDN.Compared with the K-Nearest Neighbor(K-NN),Artificial Neural Network(ANN)models,Support Vector Machine(SVM)and Random Forest models,our proposed method outperforms in accuracy,precision and F1 values.

Data Secure Storage Mechanism of Sensor Networks Based on Blockchain

As the number of sensor network application scenarios continues to grow,the security problems inherent in this approach have become obstacles that hinder its wide application.However,it has attracted increasing attention from industry and academia.The blockchain is based on a distributed network and has the characteristics of non-tampering and traceability of block data.It is thus naturally able to solve the security problems of the sensor networks.Accordingly,this paper first analyzes the security risks associated with data storage in the sensor networks,then proposes using blockchain technology to ensure that data storage in the sensor networks is secure.In the traditional blockchain,the data layer uses a Merkle hash tree to store data;however,the Merkle hash tree cannot provide non-member proof,which makes it unable to resist the attacks of malicious nodes in networks.To solve this problem,this paper utilizes a cryptographic accumulator rather than a Merkle hash tree to provide both member proof and non-member proof.Moreover,the number of elements in the existing accumulator is limited and unable to meet the blockchain’s expansion requirements.This paper therefore proposes a new type of unbounded accumulator and provides its definition and security model.Finally,this paper constructs an unbounded accumulator scheme using bilinear pairs and analyzes its performance.

A Position-Aware Transformer for Image Captioning

Image captioning aims to generate a corresponding description of an image.In recent years,neural encoder-decodermodels have been the dominant approaches,in which the Convolutional Neural Network(CNN)and Long Short TermMemory(LSTM)are used to translate an image into a natural language description.Among these approaches,the visual attention mechanisms are widely used to enable deeper image understanding through fine-grained analysis and even multiple steps of reasoning.However,most conventional visual attention mechanisms are based on high-level image features,ignoring the effects of other image features,and giving insufficient consideration to the relative positions between image features.In this work,we propose a Position-Aware Transformer model with image-feature attention and position-aware attention mechanisms for the above problems.The image-feature attention firstly extracts multi-level features by using Feature Pyramid Network(FPN),then utilizes the scaled-dot-product to fuse these features,which enables our model to detect objects of different scales in the image more effectivelywithout increasing parameters.In the position-aware attentionmechanism,the relative positions between image features are obtained at first,afterwards the relative positions are incorporated into the original image features to generate captions more accurately.Experiments are carried out on the MSCOCO dataset and our approach achieves competitive BLEU-4,METEOR,ROUGE-L,CIDEr scores compared with some state-of-the-art approaches,demonstrating the effectiveness of our approach.

A Reliable and Scalable Internet of Military Things Architecture

Recently,Internet of Things(IoT)technology has provided logistics services to many disciplines such as agriculture,industry,and medicine.Thus,it has become one of the most important scientific research fields.Applying IoT to military domain has many challenges such as fault tolerance and QoS.In this paper,IoT technology is applied on the military field to create an Internet of Military Things(IoMT)system.Here,the architecture of the aforementioned IoMT system is proposed.This architecture consists of four main layers:Communication,information,application,and decision support.These layers provided a fault tolerant coverage communication system for IoMT things.Moreover,it implemented data reduction methods such as filtering,compression,abstraction,and data prioritization queuing system to guarantee QoS for the transmitted data.Furthermore,it used decision support technology and IoMT application unification ideas.Finally,to evaluate the IoMT system,an intensive simulation environment is constructed using the network simulation package,NS3.The simulation results proved that the proposed IoMT system outperformed the traditional military system with regard to the performance metrics,packet loss,end-to-end delay,throughput,energy consumption ratio,and data reduction rate.

A Fast and Effective Multiple Kernel Clustering Method on Incomplete Data

Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled.However,multiple kernel clustering for incomplete data is a critical yet challenging task.Although the existing absent multiple kernel clustering methods have achieved remarkable performance on this task,they may fail when data has a high value-missing rate,and they may easily fall into a local optimum.To address these problems,in this paper,we propose an absent multiple kernel clustering(AMKC)method on incomplete data.The AMKC method rst clusters the initialized incomplete data.Then,it constructs a new multiple-kernel-based data space,referred to as K-space,from multiple sources to learn kernel combination coefcients.Finally,it seamlessly integrates an incomplete-kernel-imputation objective,a multiple-kernel-learning objective,and a kernel-clustering objective in order to achieve absent multiple kernel clustering.The three stages in this process are carried out simultaneously until the convergence condition is met.Experiments on six datasets with various characteristics demonstrate that the kernel imputation and clustering performance of the proposed method is signicantly better than state-of-the-art competitors.Meanwhile,the proposed method gains fast convergence speed.

A Sustainable WSN System with Heuristic Schemes in IIoT

Recently, the development of Industrial Internet of Things hastaken the advantage of 5G network to be more powerful and more intelligent.However, the upgrading of 5G network will cause a variety of issues increase,one of them is the increased cost of coverage. In this paper, we proposea sustainable wireless sensor networks system, which avoids the problemsbrought by 5G network system to some extent. In this system, deployingrelays and selecting routing are for the sake of communication and charging.The main aim is to minimize the total energy-cost of communication underthe precondition, where each terminal with low-power should be charged byat least one relay. Furthermore, from the perspective of graph theory, weextract a combinatorial optimization problem from this system. After that,as to four different cases, there are corresponding different versions of theproblem. We give the proofs of computational complexity for these problems,and two heuristic algorithms for one of them are proposed. Finally, theextensive experiments compare and demonstrate the performances of thesetwo algorithms.