Task-Oriented Semantic Communication with Foundation Models

In the future development direction of the sixth generation(6G)mobile communication,several communication models are proposed to face the growing challenges of the task.The rapid development of artificial intelligence(AI)foundation models provides significant support for efficient and intelligent communication interactions.In this paper,we propose an innovative semantic communication paradigm called task-oriented semantic communication system with foundation models.First,we segment the image by using task prompts based on the segment anything model(SAM)and contrastive language-image pretraining(CLIP).Meanwhile,we adopt Bezier curve to enhance the mask to improve the segmentation accuracy.Second,we have differentiated semantic compression and transmission approaches for segmented content.Third,we fuse different semantic information based on the conditional diffusion model to generate high-quality images that satisfy the users'specific task requirements.Finally,the experimental results show that the proposed system compresses the semantic information effectively and improves the robustness of semantic communication.

PCB CT Image Element Segmentation Model Optimizing the Semantic Perception of Connectivity Relationship

Computed Tomography(CT)is a commonly used technology in Printed Circuit Boards(PCB)non-destructive testing,and element segmentation of CT images is a key subsequent step.With the development of deep learning,researchers began to exploit the“pre-training and fine-tuning”training process for multi-element segmentation,reducing the time spent on manual annotation.However,the existing element segmentation model only focuses on the overall accuracy at the pixel level,ignoring whether the element connectivity relationship can be correctly identified.To this end,this paper proposes a PCB CT image element segmentation model optimizing the semantic perception of connectivity relationship(OSPC-seg).The overall training process adopts a“pre-training and fine-tuning”training process.A loss function that optimizes the semantic perception of circuit connectivity relationship(OSPC Loss)is designed from the aspect of alleviating the class imbalance problem and improving the correct connectivity rate.Also,the correct connectivity rate index(CCR)is proposed to evaluate the model’s connectivity relationship recognition capabilities.Experiments show that mIoU and CCR of OSPC-seg on our datasets are 90.1%and 97.0%,improved by 1.5%and 1.6%respectively compared with the baseline model.From visualization results,it can be seen that the segmentation performance of connection positions is significantly improved,which also demonstrates the effectiveness of OSPC-seg.

Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

Intellicise Model Transmission for Semantic Communication in Intelligence-Native 6G Networks

To facilitate emerging applications and demands of edge intelligence(EI)-empowered 6G networks,model-driven semantic communications have been proposed to reduce transmission volume by deploying artificial intelligence(AI)models that provide abilities of semantic extraction and recovery.Nevertheless,it is not feasible to preload all AI models on resource-constrained terminals.Thus,in-time model transmission becomes a crucial problem.This paper proposes an intellicise model transmission architecture to guarantee the reliable transmission of models for semantic communication.The mathematical relationship between model size and performance is formulated by employing a recognition error function supported with experimental data.We consider the characteristics of wireless channels and derive the closed-form expression of model transmission outage probability(MTOP)over the Rayleigh channel.Besides,we define the effective model accuracy(EMA)to evaluate the model transmission performance of both communication and intelligence.Then we propose a joint model selection and resource allocation(JMSRA)algorithm to maximize the average EMA of all users.Simulation results demonstrate that the average EMA of the JMSRA algorithm outperforms baseline algorithms by about 22%.

Parameter-driven Level of Detail Derivation Method for Semantic Building Facade Model

The relentless progress in the research of geographic spatial data models and their application scenarios is propelling an unprecedented rich Level of Detail(LoD)in realistic 3D representation and smart cities.This pursuit of rich details not only adds complexity to entity models but also poses significant computational challenges for model visualization and 3D GIS.This paper introduces a novel method for deriving multi-LOD models,which can enhance the efficiency of spatial computing in complex 3D building models.Firstly,we extract multiple facades from a 3D building model(LoD3)and convert them into individual semantic facade models.Through the utilization of the developed facade layout graph,each semantic facade model is then transformed into a parametric model.Furthermore,we explore the specification of geometric and semantic details in building facades and define three different LODs for facades,offering a unique expression.Finally,an innovative heuristic method is introduced to simplify the parameterized facade.Through rigorous experimentation and evaluation,the effectiveness of the proposed parameterization methodology in capturing complex geometric details,semantic richness,and topological relationships of 3D building models is demonstrated.

Large Language Model Based Semantic Parsing for Intelligent Database Query Engine

With the rapid development of artificial intelligence, large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. These models have great potential to enhance database query systems, enabling more intuitive and semantic query mechanisms. Our model leverages LLM’s deep learning architecture to interpret and process natural language queries and translate them into accurate database queries. The system integrates an LLM-powered semantic parser that translates user input into structured queries that can be understood by the database management system. First, the user query is pre-processed, the text is normalized, and the ambiguity is removed. This is followed by semantic parsing, where the LLM interprets the pre-processed text and identifies key entities and relationships. This is followed by query generation, which converts the parsed information into a structured query format and tailors it to the target database schema. Finally, there is query execution and feedback, where the resulting query is executed on the database and the results are returned to the user. The system also provides feedback mechanisms to improve and optimize future query interpretations. By using advanced LLMs for model implementation and fine-tuning on diverse datasets, the experimental results show that the proposed method significantly improves the accuracy and usability of database queries, making data retrieval easy for users without specialized knowledge.

Variational Learned Talking-Head Semantic Coded Transmission System

Video transmission requires considerable bandwidth,and current widely employed schemes prove inadequate when confronted with scenes featuring prominently.Motivated by the strides in talkinghead generative technology,the paper introduces a semantic transmission system tailored for talking-head videos.The system captures semantic information from talking-head video and faithfully reconstructs source video at the receiver,only one-shot reference frame and compact semantic features are required for the entire transmission.Specifically,we analyze video semantics in the pixel domain frame-by-frame and jointly process multi-frame semantic information to seamlessly incorporate spatial and temporal information.Variational modeling is utilized to evaluate the diversity of importance among group semantics,thereby guiding bandwidth resource allocation for semantics to enhance system efficiency.The whole endto-end system is modeled as an optimization problem and equivalent to acquiring optimal rate-distortion performance.We evaluate our system on both reference frame and video transmission,experimental results demonstrate that our system can improve the efficiency and robustness of communications.Compared to the classical approaches,our system can save over 90%of bandwidth when user perception is close.

Information Conductivity:Universal Performance Measure for Semantic Communications

As a novel paradigm,semantic communication provides an effective solution for breaking through the future development dilemma of classical communication systems.However,it remains an unsolved problem of how to measure the information transmission capability for a given semantic communication method and subsequently compare it with the classical communication method.In this paper,we first present a review of the semantic communication system,including its system model and the two typical coding and transmission methods for its implementations.To address the unsolved issue of the information transmission capability measure for semantic communication methods,we propose a new universal performance measure called Information Conductivity.We provide the definition and the physical significance to state its effectiveness in representing the information transmission capabilities of the semantic communication systems and present elaborations including its measure methods,degrees of freedom,and progressive analysis.Experimental results in image transmission scenarios validate its practical applicability.

SHEL:a semantically enhanced hardware-friendly entity linking method

With the help of pre-trained language models,the accuracy of the entity linking task has made great strides in recent years.However,most models with excellent performance require fine-tuning on a large amount of training data using large pre-trained language models,which is a hardware threshold to accomplish this task.Some researchers have achieved competitive results with less training data through ingenious methods,such as utilizing information provided by the named entity recognition model.This paper presents a novel semantic-enhancement-based entity linking approach,named semantically enhanced hardware-friendly entity linking(SHEL),which is designed to be hardware friendly and efficient while maintaining good performance.Specifically,SHEL's semantic enhancement approach consists of three aspects:(1)semantic compression of entity descriptions using a text summarization model;(2)maximizing the capture of mention contexts using asymmetric heuristics;(3)calculating a fixed size mention representation through pooling operations.These series of semantic enhancement methods effectively improve the model's ability to capture semantic information while taking into account the hardware constraints,and significantly improve the model's convergence speed by more than 50%compared with the strong baseline model proposed in this paper.In terms of performance,SHEL is comparable to the previous method,with superior performance on six well-established datasets,even though SHEL is trained using a smaller pre-trained language model as the encoder.

A Semantic-Sensitive Approach to Indoor and Outdoor 3D Data Organization

Building model data organization is often programmed to solve a specific problem,resulting in the inability to organize indoor and outdoor 3D scenes in an integrated manner.In this paper,existing building spatial data models are studied,and the characteristics of building information modeling standards(IFC),city geographic modeling language(CityGML),indoor modeling language(IndoorGML),and other models are compared and analyzed.CityGML and IndoorGML models face challenges in satisfying diverse application scenarios and requirements due to limitations in their expression capabilities.It is proposed to combine the semantic information of the model objects to effectively partition and organize the indoor and outdoor spatial 3D model data and to construct the indoor and outdoor data organization mechanism of“chunk-layer-subobject-entrances-area-detail object.”This method is verified by proposing a 3D data organization method for indoor and outdoor space and constructing a 3D visualization system based on it.

Configuration knowledge modeling of customizable products based on semantic web technologies

In order to solve the problem of modeling product configuration knowledge at the semantic level to successfully implement the mass customization strategy, an approach of ontology-based configuration knowledge modeling, combining semantic web technologies, was proposed. A general configuration ontology was developed to provide a common concept structure for modeling configuration knowledge and rules of specific product domains. The OWL web ontology language and semantic web rule language （SWRL） were used to formally represent the configuration ontology, domain configuration knowledge and rules to enhance the consistency, maintainability and reusability of all the configuration knowledge. The configuration knowledge modeling of a customizable personal computer family shows that the approach can provide explicit, computerunderstandable knowledge semantics for specific product configuration domains and can efficiently support automatic configuration tasks of complex products.

Word Embeddings and Semantic Spaces in Natural Language Processing

One of the critical hurdles, and breakthroughs, in the field of Natural Language Processing (NLP) in the last two decades has been the development of techniques for text representation that solves the so-called curse of dimensionality, a problem which plagues NLP in general given that the feature set for learning starts as a function of the size of the language in question, upwards of hundreds of thousands of terms typically. As such, much of the research and development in NLP in the last two decades has been in finding and optimizing solutions to this problem, to feature selection in NLP effectively. This paper looks at the development of these various techniques, leveraging a variety of statistical methods which rest on linguistic theories that were advanced in the middle of the last century, namely the distributional hypothesis which suggests that words that are found in similar contexts generally have similar meanings. In this survey paper we look at the development of some of the most popular of these techniques from a mathematical as well as data structure perspective, from Latent Semantic Analysis to Vector Space Models to their more modern variants which are typically referred to as word embeddings. In this review of algoriths such as Word2Vec, GloVe, ELMo and BERT, we explore the idea of semantic spaces more generally beyond applicability to NLP.

Spatio-temporal GIS Data Model Based on Event Semantics

There are mainly four kinds of models to record and deal with historical information. By taking them as reference, the spatio-temporal model based on event semantics is proposed. In this model, according to the way for describing an event, all the information are divided into five domains. This paper describes the model by using the land parcel change in the cadastral information system, and expounds the model by using five tables corresponding to the five domains. With the aid of this model, seven examples are given on historical query, trace back and recurrence. This model can be implemented either in the extended relational database or in the object-oriented database.

A Novel Cross-Media Layered Semantic Mining Model

This paper presents a cross-media semantic mining model （CSMM） based on object semantic. This model obtains object-level semantic information in terms of maximum probability principle. Then semantic templates are trained and constructed with STTS （Semantic Template Training System）, which are taken as the bridge to realize the transition from various low-level media feature to object semantic. Furthermore, we put forward a kind of double layers metadata structure to efficaciously store and manage mined low-level feature and high-level semantic. This model has broad application in lots of domains such as intelligent retrieval engine, medical diagnoses, multimedia design and so on.

Annotation and Retrieval System of CAD Models Based on Functional Semantics

CAD model retrieval based on functional semantics is more significant than content-based 3D model retrieval during the mechanical conceptual design phase. However, relevant research is still not fully discussed. Therefore, a functional semantic-based CAD model annotation and retrieval method is proposed to support mechanical conceptual design and design reuse, inspire designer creativity through existing CAD models, shorten design cycle, and reduce costs. Firstly, the CAD model functional semantic ontology is constructed to formally represent the functional semantics of CAD models and describe the mechanical conceptual design space comprehensively and consistently. Secondly, an approach to represent CAD models as attributed adjacency graphs（AAG） is proposed. In this method, the geometry and topology data are extracted from STEP models. On the basis of AAG, the functional semantics of CAD models are annotated semi-automatically by matching CAD models that contain the partial features of which functional semantics have been annotated manually, thereby constructing CAD Model Repository that supports model retrieval based on functional semantics. Thirdly, a CAD model retrieval algorithm that supports multi-function extended retrieval is proposed to explore more potential creative design knowledge in the semantic level. Finally, a prototype system, called Functional Semantic-based CAD Model Annotation and Retrieval System（FSMARS）, is implemented. A case demonstrates that FSMARS can successfully botain multiple potential CAD models that conform to the desired function. The proposed research addresses actual needs and presents a new way to acquire CAD models in the mechanical conceptual design phase.

Web Service Description and Discovery Based on Semantic Model

A novel semantic model of Web service descrip tion and discovery was proposed through an extension for profile model of Web ontology language for services （OWL-S） in this paper. Similarity matching of Web services was implemented through computing weighted summation of semantic similarity value based on specific domain ontology and dynamical satisfy extent evaluation for quality of service （QoS）. Experiments show that the provided semantic matching model is efficient.

Semantic image annotation based on GMM and random walk model

Automatic image annotation has been an active topic of research in computer vision and pattern recognition for decades.A two stage automatic image annotation method based on Gaussian mixture model(GMM) and random walk model(abbreviated as GMM-RW) is presented.To start with,GMM fitted by the rival penalized expectation maximization(RPEM) algorithm is employed to estimate the posterior probabilities of each annotation keyword.Subsequently,a random walk process over the constructed label similarity graph is implemented to further mine the potential correlations of the candidate annotations so as to capture the refining results,which plays a crucial role in semantic based image retrieval.The contributions exhibited in this work are multifold.First,GMM is exploited to capture the initial semantic annotations,especially the RPEM algorithm is utilized to train the model that can determine the number of components in GMM automatically.Second,a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels,which is able to avoid the phenomena of polysemy and synonym efficiently during the image annotation process.Third,the random walk is implemented over the constructed label graph to further refine the candidate set of annotations generated by GMM.Conducted experiments on the standard Corel5 k demonstrate that GMM-RW is significantly more effective than several state-of-the-arts regarding their effectiveness and efficiency in the task of automatic image annotation.

Unstructured Road Extraction in UAV Images based on Lightweight Model

There is no unified planning standard for unstructured roads,and the morphological structures of these roads are complex and varied.It is important to maintain a balance between accuracy and speed for unstructured road extraction models.Unstructured road extraction algorithms based on deep learning have problems such as high model complexity,high computational cost,and the inability to adapt to current edge computing devices.Therefore,it is best to use lightweight network models.Considering the need for lightweight models and the characteristics of unstructured roads with different pattern shapes,such as blocks and strips,a TMB(Triple Multi-Block)feature extraction module is proposed,and the overall structure of the TMBNet network is described.The TMB module was compared with SS-nbt,Non-bottleneck-1D,and other modules via experiments.The feasibility and effectiveness of the TMB module design were proven through experiments and visualizations.The comparison experiment,using multiple convolution kernel categories,proved that the TMB module can improve the segmentation accuracy of the network.The comparison with different semantic segmentation networks demonstrates that the TMBNet network has advantages in terms of unstructured road extraction.

AI-infused Semantic Model to Enrich and Expand Programming Question Generation

Creating practice questions for programming learning is not easy.It requires the instructor to diligently organize heterogeneous learning resources,that is,conceptual programming concepts and procedural programming rules.Today’s programming question generation(PQG)is still largely relying on the demanding creation task performed by the instructors without advanced technological support.In this work,we propose a semantic PQG model that aims to help the instructor generate new programming questions and expand the assessment items.The PQG model is designed to transform conceptual and procedural programming knowledge from textbooks into a semantic network by the Local Knowledge Graph(LKG)and Abstract Syntax Tree(AST).For any given question,the model queries the established network to find related code examples and generates a set of questions by the associated LKG/AST semantic structures.We conduct analysis to compare instructor-made questions from 9 undergraduate introductory programming courses and textbook questions.The results show that the instructormade questions had much simpler complexity than the textbook ones.The disparity of topic distribution intrigued us to further research the breadth and depth of question quality and also to investigate the complexity of the questions in relation to the student performances.Finally,we report a user study results on the proposed Artificial Intelligent-infused semantic PQG model in examining the machine-generated questions’quality.

Developing Semantic Business Model for VO Construction on Semantic Grid

This paper combines semantic web technology with business modeling and yields semantic business model that is semantically described in terms of roles and relationships. The semantic business model can be used to discover grid services by means of automation tools. The gap between business goals and grid services is bridged by role relationships and compositions of them, so that the virtual organization evolution is supported effectively. Semantic business model can support virtual organization validation at design stage rather than at run-time stage. The designers can animate their business model and make initial assessment of what interactions should occur between roles and in which order. The users can verify whether the grid service compositions satisfy business goals.