Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the...Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the research and applications of natural language processing across different modalities,our goal is to accurately extract frame-level semantic information from videos and ultimately transmit high-quality videos.Specifically,we propose a deep learning-basedMulti-ModalMutual Enhancement Video Semantic Communication system,called M3E-VSC.Built upon a VectorQuantized Generative AdversarialNetwork(VQGAN),our systemaims to leverage mutual enhancement among different modalities by using text as the main carrier of transmission.With it,the semantic information can be extracted fromkey-frame images and audio of the video and performdifferential value to ensure that the extracted text conveys accurate semantic information with fewer bits,thus improving the capacity of the system.Furthermore,a multi-frame semantic detection module is designed to facilitate semantic transitions during video generation.Simulation results demonstrate that our proposed model maintains high robustness in complex noise environments,particularly in low signal-to-noise ratio conditions,significantly improving the accuracy and speed of semantic transmission in video communication by approximately 50 percent.展开更多
In the information age,blended teaching,no matter online or offline,has become the mainstream of college teaching reform.In this teaching model,self-directed learning and cooperative learning are the two main learning...In the information age,blended teaching,no matter online or offline,has become the mainstream of college teaching reform.In this teaching model,self-directed learning and cooperative learning are the two main learning approaches.On the online teaching platform,students mainly learn knowledge-based content by self-directed learning,while practising their language skills by cooperative learning in flipped classroom activities.On one hand,it advocates student-centered strategy so as to improve students autonomous learning ability;on the other hand,teachers serve as a guide to organize the classroom activities;meanwhile,they give timely feedback to students in order to promote students’learning ability.In blended teaching model,this mutually compatible and reinforcing model of self-directed learning and cooperative learning is undoubtedly helpful to improve the teaching efficiency.展开更多
基金supported by the National Key Research and Development Project under Grant 2020YFB1807602Key Program of Marine Economy Development Special Foundation of Department of Natural Resources of Guangdong Province(GDNRC[2023]24)the National Natural Science Foundation of China under Grant 62271267.
文摘Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the research and applications of natural language processing across different modalities,our goal is to accurately extract frame-level semantic information from videos and ultimately transmit high-quality videos.Specifically,we propose a deep learning-basedMulti-ModalMutual Enhancement Video Semantic Communication system,called M3E-VSC.Built upon a VectorQuantized Generative AdversarialNetwork(VQGAN),our systemaims to leverage mutual enhancement among different modalities by using text as the main carrier of transmission.With it,the semantic information can be extracted fromkey-frame images and audio of the video and performdifferential value to ensure that the extracted text conveys accurate semantic information with fewer bits,thus improving the capacity of the system.Furthermore,a multi-frame semantic detection module is designed to facilitate semantic transitions during video generation.Simulation results demonstrate that our proposed model maintains high robustness in complex noise environments,particularly in low signal-to-noise ratio conditions,significantly improving the accuracy and speed of semantic transmission in video communication by approximately 50 percent.
文摘In the information age,blended teaching,no matter online or offline,has become the mainstream of college teaching reform.In this teaching model,self-directed learning and cooperative learning are the two main learning approaches.On the online teaching platform,students mainly learn knowledge-based content by self-directed learning,while practising their language skills by cooperative learning in flipped classroom activities.On one hand,it advocates student-centered strategy so as to improve students autonomous learning ability;on the other hand,teachers serve as a guide to organize the classroom activities;meanwhile,they give timely feedback to students in order to promote students’learning ability.In blended teaching model,this mutually compatible and reinforcing model of self-directed learning and cooperative learning is undoubtedly helpful to improve the teaching efficiency.
