Semantic segmentation-based semantic communication system for image transmission

With the rapid development of artificial intelligence and the widespread use of the Internet of Things, semantic communication, as an emerging communication paradigm, has been attracting great interest. Taking image transmission as an example, from the semantic communication's view, not all pixels in the images are equally important for certain receivers. The existing semantic communication systems directly perform semantic encoding and decoding on the whole image, in which the region of interest cannot be identified. In this paper, we propose a novel semantic communication system for image transmission that can distinguish between Regions Of Interest (ROI) and Regions Of Non-Interest (RONI) based on semantic segmentation, where a semantic segmentation algorithm is used to classify each pixel of the image and distinguish ROI and RONI. The system also enables high-quality transmission of ROI with lower communication overheads by transmissions through different semantic communication networks with different bandwidth requirements. An improved metric θPSNR is proposed to evaluate the transmission accuracy of the novel semantic transmission network. Experimental results show that our proposed system achieves a significant performance improvement compared with existing approaches, namely, existing semantic communication approaches and the conventional approach without semantics.

A Natural Language Generation Algorithm for Greek by Using Hole Semantics and a Systemic Grammatical Formalism

This work is about the progress of previous related work based on an experiment to improve the intelligence of robotic systems,with the aim of achieving more linguistic communication capabilities between humans and robots.In this paper,the authors attempt an algorithmic approach to natural language generation through hole semantics and by applying the OMAS-III computational model as a grammatical formalism.In the original work,a technical language is used,while in the later works,this has been replaced by a limited Greek natural language dictionary.This particular effort was made to give the evolving system the ability to ask questions,as well as the authors developed an initial dialogue system using these techniques.The results show that the use of these techniques the authors apply can give us a more sophisticated dialogue system in the future.

Using ontology and rules to retrieve the semantics of disaster remote sensing data

Remote sensing data plays an important role in natural disaster management.However,with the increase of the variety and quantity of remote sensors,the problem of“knowledge barriers”arises when data users in disaster field retrieve remote sensing data.To improve this problem,this paper proposes an ontology and rule based retrieval(ORR)method to retrieve disaster remote sensing data,and this method introduces ontology technology to express earthquake disaster and remote sensing knowledge,on this basis,and realizes the task suitability reasoning of earthquake disaster remote sensing data,mining the semantic relationship between remote sensing metadata and disasters.The prototype system is built according to the ORR method,which is compared with the traditional method,using the ORR method to retrieve disaster remote sensing data can reduce the knowledge requirements of data users in the retrieval process and improve data retrieval efficiency.

Variational Learned Talking-Head Semantic Coded Transmission System

Video transmission requires considerable bandwidth,and current widely employed schemes prove inadequate when confronted with scenes featuring prominently.Motivated by the strides in talkinghead generative technology,the paper introduces a semantic transmission system tailored for talking-head videos.The system captures semantic information from talking-head video and faithfully reconstructs source video at the receiver,only one-shot reference frame and compact semantic features are required for the entire transmission.Specifically,we analyze video semantics in the pixel domain frame-by-frame and jointly process multi-frame semantic information to seamlessly incorporate spatial and temporal information.Variational modeling is utilized to evaluate the diversity of importance among group semantics,thereby guiding bandwidth resource allocation for semantics to enhance system efficiency.The whole endto-end system is modeled as an optimization problem and equivalent to acquiring optimal rate-distortion performance.We evaluate our system on both reference frame and video transmission,experimental results demonstrate that our system can improve the efficiency and robustness of communications.Compared to the classical approaches,our system can save over 90%of bandwidth when user perception is close.

Building Semantic Communication System via Molecules:An End-to-End Training Approach

The concept of semantic communication provides a novel approach for applications in scenarios with limited communication resources.In this paper,we propose an end-to-end(E2E)semantic molecular communication system,aiming to enhance the efficiency of molecular communication systems by reducing the transmitted information.Specifically,following the joint source channel coding paradigm,the network is designed to encode the task-relevant information into the concentration of the information molecules,which is robust to the degradation of the molecular communication channel.Furthermore,we propose a channel network to enable the E2E learning over the non-differentiable molecular channel.Experimental results demonstrate the superior performance of the semantic molecular communication system over the conventional methods in classification tasks.

Enhancing Deep Learning Semantics:The Diffusion Sampling and Label-Driven Co-Attention Approach

The advent of self-attention mechanisms within Transformer models has significantly propelled the advancement of deep learning algorithms,yielding outstanding achievements across diverse domains.Nonetheless,self-attention mechanisms falter when applied to datasets with intricate semantic content and extensive dependency structures.In response,this paper introduces a Diffusion Sampling and Label-Driven Co-attention Neural Network(DSLD),which adopts a diffusion sampling method to capture more comprehensive semantic information of the data.Additionally,themodel leverages the joint correlation information of labels and data to introduce the computation of text representation,correcting semantic representationbiases in thedata,andincreasing the accuracyof semantic representation.Ultimately,the model computes the corresponding classification results by synthesizing these rich data semantic representations.Experiments on seven benchmark datasets show that our proposed model achieves competitive results compared to state-of-the-art methods.

Extended context-based semantic communication system for text transmission

Context information is significant for semantic extraction and recovery of messages in semantic communication.However,context information is not fully utilized in the existing semantic communication systems since re-lationships between sentences are often ignored.In this paper,we propose an Extended Context-based Semantic Communication(ECSC)system for text transmission,in which context information within and between sentences is explored for semantic representation and recovery.At the encoder,self-attention and segment-level relative attention are used to extract context information within and between sentences,respectively.In addition,a gate mechanism is adopted at the encoder to incorporate the context information from different ranges.At the decoder,Transformer-XL is introduced to obtain more semantic information from the historical communication processes for semantic recovery.Simulation results show the effectiveness of our proposed model in improving the semantic accuracy between transmitted and recovered messages under various channel conditions.

Variational Neural Inference Enhanced Text Semantic Communication System

Recently,deep learning-based semantic communication has garnered widespread attention,with numerous systems designed for transmitting diverse data sources,including text,image,and speech,etc.While efforts have been directed toward improving system performance,many studies have concentrated on enhancing the structure of the encoder and decoder.However,this often overlooks the resulting increase in model complexity,imposing additional storage and computational burdens on smart devices.Furthermore,existing work tends to prioritize explicit semantics,neglecting the potential of implicit semantics.This paper aims to easily and effectively enhance the receiver's decoding capability without modifying the encoder and decoder structures.We propose a novel semantic communication system with variational neural inference for text transmission.Specifically,we introduce a simple but effective variational neural inferer at the receiver to infer the latent semantic information within the received text.This information is then utilized to assist in the decoding process.The simulation results show a significant enhancement in system performance and improved robustness.

A general Boolean semantic modelling approach for complex and intelligent industrial systems in the framework of DES

Discrete event system(DES)models promote system engineering,including system design,verification,and assessment.The advancement in manufacturing technology has endowed us to fabricate complex industrial systems.Consequently,the adoption of advanced modeling methodologies adept at handling complexity and scalability is imperative.Moreover,industrial systems are no longer quiescent,thus the intelligent operations of the systems should be dynamically specified in the model.In this paper,the composition of the subsystem behaviors is studied to generate the complexity and scalability of the global system model,and a Boolean semantic specifying algorithm is proposed for generating dynamic intelligent operations in the model.In traditional modeling approaches,the change or addition of specifications always necessitates the complete resubmission of the system model,a resource-consuming and error-prone process.Compared with traditional approaches,our approach has three remarkable advantages:(i)an established Boolean semantic can be fitful for all kinds of systems;(ii)there is no need to resubmit the system model whenever there is a change or addition of the operations;(iii)multiple specifying tasks can be easily achieved by continuously adding a new semantic.Thus,this general modeling approach has wide potential for future complex and intelligent industrial systems.

Efficient Vision Transformers for Autonomous Off-Road Perception Systems

The development of autonomous vehicles has become one of the greatest research endeavors in recent years. These vehicles rely on many complex systems working in tandem to make decisions. For practical use and safety reasons, these systems must not only be accurate, but also quickly detect changes in the surrounding environment. In autonomous vehicle research, the environment perception system is one of the key components of development. Environment perception systems allow the vehicle to understand its surroundings. This is done by using cameras, light detection and ranging (LiDAR), with other sensor systems and modalities. Deep learning computer vision algorithms have been shown to be the strongest tool for translating camera data into accurate and safe traversability decisions regarding the environment surrounding a vehicle. In order for a vehicle to safely traverse an area in real time, these computer vision algorithms must be accurate and have low latency. While much research has studied autonomous driving for traversing well-structured urban environments, limited research exists evaluating perception system improvements in off-road settings. This research aims to investigate the adaptability of several existing deep-learning architectures for semantic segmentation in off-road environments. Previous studies of two Convolutional Neural Network (CNN) architectures are included for comparison with new evaluation of Vision Transformer (ViT) architectures for semantic segmentation. Our results demonstrate viability of ViT architectures for off-road perception systems, having a strong segmentation accuracy, lower inference speed and memory footprint compared to previous results with CNN architectures.

Query Expansion Based on Semantics and Statistics in Chinese Question Answering System

In Chinese question answering system, because there is more semantic relation in questions than that in query words, the precision can be improved by expanding query while using natural language questions to retrieve documents. This paper proposes a new approach to query expansion based on semantics and statistics Firstly automatic relevance feedback method is used to generate a candidate expansion word set. Then the expanded query words are selected from the set based on the semantic similarity and seman- tic relevancy between the candidate words and the original words. Experiments show the new approach is effective for Web retrieval and out-performs the conventional expansion approaches.

Annotation and Retrieval System of CAD Models Based on Functional Semantics

CAD model retrieval based on functional semantics is more significant than content-based 3D model retrieval during the mechanical conceptual design phase. However, relevant research is still not fully discussed. Therefore, a functional semantic-based CAD model annotation and retrieval method is proposed to support mechanical conceptual design and design reuse, inspire designer creativity through existing CAD models, shorten design cycle, and reduce costs. Firstly, the CAD model functional semantic ontology is constructed to formally represent the functional semantics of CAD models and describe the mechanical conceptual design space comprehensively and consistently. Secondly, an approach to represent CAD models as attributed adjacency graphs（AAG） is proposed. In this method, the geometry and topology data are extracted from STEP models. On the basis of AAG, the functional semantics of CAD models are annotated semi-automatically by matching CAD models that contain the partial features of which functional semantics have been annotated manually, thereby constructing CAD Model Repository that supports model retrieval based on functional semantics. Thirdly, a CAD model retrieval algorithm that supports multi-function extended retrieval is proposed to explore more potential creative design knowledge in the semantic level. Finally, a prototype system, called Functional Semantic-based CAD Model Annotation and Retrieval System（FSMARS）, is implemented. A case demonstrates that FSMARS can successfully botain multiple potential CAD models that conform to the desired function. The proposed research addresses actual needs and presents a new way to acquire CAD models in the mechanical conceptual design phase.

Approximate Failures Semantics for Polynomial Labelled Transition Systems

Labelled transition systems(LTSs) are widely used to formally describe system behaviour.The labels of LTS are extended to offer a more satisfactory description of behaviour by refining the abstract labels into multivariate polynomials.These labels can be simplified by numerous numerical approximation methods.Those LTSs that can not apply failures semantics equivalence in description and verification may have a chance after using approximation on labels.The technique that combines approximation and failures semantics equivalence effectively alleviates the computational complexity and minimizes LTS.

Lightweight Network Ensemble Architecture for Environmental Perception on the Autonomous System

It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.

Ontology-Based Crime News Semantic Retrieval System

Every day,the media reports tons of crimes that are considered by a large number of users and accumulate on a regular basis.Crime news exists on the Internet in unstructured formats such as books,websites,documents,and journals.From such homogeneous data,it is very challenging to extract relevant information which is a time-consuming and critical task for the public and law enforcement agencies.Keyword-based Information Retrieval(IR)systems rely on statistics to retrieve results,making it difficult to obtain relevant results.They are unable to understandthe user’s query and thus facewordmismatchesdue to context changes andthe inevitable semanticsof a given word.Therefore,such datasets need to be organized in a structured configuration,with the goal of efficiently manipulating the data while respecting the semantics of the data.An ontological semantic IR systemis needed that can find the right investigative information and find important clues to solve criminal cases.The semantic system retrieves information in view of the similarity of the semantics among indexed data and user queries.In this paper,we develop anontology-based semantic IRsystemthat leverages the latest semantic technologies including resource description framework(RDF),semantic protocol and RDF query language(SPARQL),semantic web rule language(SWRL),and web ontology language(OWL).We have conducted two experiments.In the first experiment,we implemented a keyword-based textual IR systemusing Apache Lucene.In the second experiment,we implemented a semantic systemthat uses ontology to store the data and retrieve precise results with high accuracy using SPARQL queries.The keyword-based system has filtered results with 51%accuracy,while the semantic system has filtered results with 95%accuracy,leading to significant improvements in the field and opening up new horizons for researchers.

Real-time collaborative design based on operation semantics in distributed CAD environment

This paper proposes a collaborative design model based on operation semantics in a distributed computer-aided design (CAD) environment. The goal is to reduce time consumption in data format conversion and the requirement of network bandwidth so as to improve the cooperative ability and the synchronization efficiency. Firstly, real-time collaborative design is reviewed and three kinds of real-time collaborative design models are discussed. Secondly, the concept of operation semantics is defined and the framework of an operation semantics model is presented. The operation semantics carries the original design data and actual operation process to express design intent and operation activity in conventional CAD systems. Finally, according to the operation semantics model, a CAD operation primitive is defined which can be retrieved from and mapped to the local CAD system operation commands; a distributed CAD collaborative architecture based on the model is presented, and an example is given to verify the model.

Searching scheme in P2P system based on semantic overlay network

In consideration of the limitation of super-peer overlay network, the semantic information was introduced into the super-peers＇ organization. A novel P2P （peer-to-peer） searching model, SSP2P, was put forward. The peers in the model were organized in a natural area autonomy system （AAS） based on the smallworld theory. A super-peer was selected in each AAS based on power law; and all the super-peers formed different super-peer semantic networks. Thus, a hierarchical super-peer overlay network was formed. The results show that the model reduces the communication cost and enhances the search efficiency while ensuring the system expansibility. It proves that the introduction of semantic information in the construction of a super-peer overlay is favorable to P2P system capability.

Web services composition based on global semantics and QoS-aware aspect

A global semantics matching and QoS-awareness service selection are proposed when aimed at a web services composition process.Both QoS-aware matching and global semantic matching are considered during the global matching.When there are demands for global semantic matching and QoS of service composition,a concrete service set which meets the demands is selected for the whole service composition process and an optimal solution is also achieved.A QoS model is built and the corresponding evaluation method is given for the matching of the service composition process.Based on them,a genetic algorithm is proposed to achieve the maximal global semantic matching degree and fulfill the QoS requirements for the whole service composition process.Experimental results and analysis show that the algorithm is feasible and effective for semantics and QoS-aware service matching.

A survey on semantic communications:Technologies,solutions,applications and challenges

Semantic Communication(SC)has emerged as a novel communication paradigm that provides a receiver with meaningful information extracted from the source to maximize information transmission throughput in wireless networks,beyond the theoretical capacity limit.Despite the extensive research on SC,there is a lack of comprehensive survey on technologies,solutions,applications,and challenges for SC.In this article,the development of SC is first reviewed and its characteristics,architecture,and advantages are summarized.Next,key technologies such as semantic extraction,semantic encoding,and semantic segmentation are discussed and their corresponding solutions in terms of efficiency,robustness,adaptability,and reliability are summarized.Applications of SC to UAV communication,remote image sensing and fusion,intelligent transportation,and healthcare are also presented and their strategies are summarized.Finally,some challenges and future research directions are presented to provide guidance for further research of SC.

Multi-User Semantic Fusion for Semantic Communications over Degraded Broadcast Channels

Degraded broadcast channels(DBC) are a typical multiuser communication scenario, Semantic communications over DBC still lack in-depth research. In this paper, we design a semantic communications approach based on multi-user semantic fusion for wireless image transmission over DBC. The transmitter extracts semantic features for two users separately and then effectively fuses them for broadcasting by leveraging semantic similarity. Unlike traditional allocation of time, power, or bandwidth, the semantic fusion scheme can dynamically control the weight of the semantic features of the two users to balance their performance. Considering the different channel state information(CSI) of both users over DBC,a DBC-Aware method is developed that embeds the CSI of both users into the joint source-channel coding encoder and fusion module to adapt to the channel.Experimental results show that the proposed system outperforms the traditional broadcasting schemes.