Rate Control Algorithm of Wireless Video Based on Visual Saliency Map Model

In order to further improve the efficiency of video compression, we introduce a perceptual characteristics of Human Visual System (HVS) to video coding, and propose a novel video coding rate control algorithm based on human visual saliency model in H.264/AVC. Firstly, we modifie Itti's saliency model. Secondly, target bits of each frame are allocated through the correlation of saliency region between the current and previous frame, and the complexity of each MB is modified through the saliency value and its Mean Absolute Difference (MAD) value. Lastly, the algorithm was implemented in JVT JM12.2. Simulation results show that, comparing with traditional rate control algorithm, the proposed one can reduce the coding bit rate and improve the reconstructed video subjective quality, especially for visual saliency region. It is very suitable for wireless video transmission.

Web-based interactive visualization of 3D video mosaics using X3D standard

We present a method of 3D image mosaicing for real 3D representation of roadside buildings, and implement a Web-based interactive visualization environment for the 3D video mosaics created by 3D image mosaicing. The 3D image mo- saicing technique developed in our previous work is a very powerful method for creating textured 3D-GIS data without excessive data processing like the laser or stereo system. For the Web-based open access to the 3D video mosaics, we build an interactive visualization environment using X3D, the emerging standard of Web 3D. We conduct the data preprocessing for 3D video mosaics and the X3D modeling for textured 3D data. The data preprocessing includes the conversion of each frame of 3D video mosaics into concatenated image files that can be hyperlinked on the Web. The X3D modeling handles the representation of concatenated images using necessary X3D nodes. By employing X3D as the data format for 3D image mosaics, the real 3D representation of roadside buildings is extended to the Web and mobile service systems.

Integrating Audio-Visual Features and Text Information for Story Segmentation of News Video

Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.

Effective Video Summarization Approach Based on Visual Attention

Video summarization is applied to reduce redundancy and developa concise representation of key frames in the video, more recently, video summaries have been used through visual attention modeling. In these schemes,the frames that stand out visually are extracted as key frames based on humanattention modeling theories. The schemes for modeling visual attention haveproven to be effective for video summaries. Nevertheless, the high cost ofcomputing in such techniques restricts their usability in everyday situations.In this context, we propose a method based on KFE (key frame extraction)technique, which is recommended based on an efficient and accurate visualattention model. The calculation effort is minimized by utilizing dynamicvisual highlighting based on the temporal gradient instead of the traditionaloptical flow techniques. In addition, an efficient technique using a discretecosine transformation is utilized for the static visual salience. The dynamic andstatic visual attention metrics are merged by means of a non-linear weightedfusion technique. Results of the system are compared with some existing stateof-the-art techniques for the betterment of accuracy. The experimental resultsof our proposed model indicate the efficiency and high standard in terms ofthe key frames extraction as output.

Video Identification Based on Watermarking Schemes and Visual Cryptography

Related to the growth of data sharing on the Internet and the wide-spread use of digital media,multimedia security and copyright protection have become of broad interest.Visual cryptography(VC)is a method of sharing a secret image between a group of participants,where certain groups of participants are defined as qualified and may combine their share of the image to obtain the original,and certain other groups are defined as prohibited,and even if they combine knowledge of their parts,they can’t obtain any information on the secret image.The visual cryptography is one of the techniques which used to transmit the secrete image under the cover picture.Human vision systems are connected to visual cryptography.The black and white image was originally used as a hidden image.In order to achieve the owner’s copy right security based on visual cryptography,a watermarking algorithm is presented.We suggest an approach in this paper to hide multiple images in video by meaningful shares using one binary share.With a common share,which we refer to as a smart key,we can decrypt several images simultaneously.Depending on a given share,the smart key decrypts several hidden images.The smart key is printed on transparency and the shares are involved in video and decryption is performed by physically superimposing the transparency on the video.Using binary,grayscale,and color images,we test the proposed method.

Analysis of the Public Visual Consumption Turn behind the Popularity of Short Video

From WeChat,QQ to Weibo,Tik Tok this series of online social networking applications,especially with the increasingly maturity of online shopping in recent years,short video and mobile intelligence have been developing rapidly,which have strongly influenced many people’s living habits and shopping habits.Their development not only promotes the social and economic development,but also produces a new form of visual communication.This paper discusses the characteristics of short video,analyzes the shift of public visual consumption behind the popularity of short video,and hopes to further build a healthy development path of short video.

Optimization of Interactive Videos Empowered the Experience of Learning Management System

The Learning management system(LMS)is now being used for uploading educational content in both distance and blended setups.LMS platform has two types of users:the educators who upload the content,and the students who have to access the content.The students,usually rely on text notes or books and video tutorials while their exams are conducted with formal methods.Formal assessments and examination criteria are ineffective with restricted learning space which makes the student tend only to read the educational contents and videos instead of interactive mode.The aim is to design an interactive LMS and examination video-based interface to cater the issues of educators and students.It is designed according to Human-computer interaction(HCI)principles to make the interactive User interface(UI)through User experience(UX).The interactive lectures in the form of annotated videos increase user engagement and improve the self-study context of users involved in LMS.The interface design defines how the design will interact with users and how the interface exchanges information.The findings show that interactive videos for LMS allow the users to have a more personalized learning experience by engaging in the educational content.The result shows a highly personalized learning experience due to the interactive video and quiz within the video.

基于计算机视觉的电力作业人员行为分析研究现状与展望

电力作业人员的有效监管是保障电力安全生产的基础。该文对电力视频中作业人员的行为识别研究进行了归类总结,涵盖静态行为分析(穿戴分析、动作分析和组合分析)和动态行为分析(复杂动作、时序行为和行为预测等);详细综述了电力作业行为分析中的核心算法模块,包括目标检测、姿态估计和视频跟踪等;论述了电力作业行为识别在算法高效性、鲁棒性、灵活性等方面所面临的应用难点和挑战,并展望了电力作业行为智能监控领域的未来发展方向,特别强调了在软硬件结合、通用大模型、生成式人工智能方面进行技术创新和改进所蕴含的潜在机会。

一种广角相机视频小卫星姿态跟踪方法

针对传统目标跟踪方法需要目标的先验位置信息、不适用于非合作目标跟踪的问题,提出了一种基于广角相机图像误差的视频小卫星目标跟踪方法。该方法首先建立了广角相机成像模型,然后计算目标在相机图像平面中的投影位置和期望位置之差,通过图像误差确定目标与视频小卫星的相对误差角速度和姿态误差,将其作为反馈项设计非线性控制器,最后利用Barbalat引理证明了系统的稳定性。仿真结果表明,在广角相机大范围视场下,该方法可以有效地将目标控制到图像平面中期望的位置,并且利用天拓二号卫星在轨验证了该方法的有效性。

长视频的超级帧切割视觉内容解释方法

针对现有基于编码解码的视频描述方法存在的对视频较长、在视频场景切换频繁情况下视觉特征提取能力不足或关键性片段捕获能力不足等视频描述不佳的问题,提出一种基于超级帧切割长视频的视频字幕方法。首先,提出超级帧提取算法,计算关键视频时间占比率以满足视频浏览时长限制,缩短视频检索时间。然后,构建两层筛选模型以自适应提取超级帧,过滤冗余关键帧,执行多场景语义描述。将保留的关键帧嵌入周围帧,利用深层网络模型以及小卷积核池化采样域获取更多的视频特征,克服了经典视频标题方法不能直接用于处理长视频的困难。最后,通过用长短时记忆模型代替循环神经网络解码生成视频标题,给出视频内容的分段解释信息。在YouTube数据集视频、合成视频和监控长视频上进行测试,采用多种机器翻译评价指标评估了该方法的性能,均获得了不同程度的提升。实验结果表明,该方法在应对视频场景切换频繁、视频较长等挑战时,能够获得较好的片段描述。

画框意识:一种视觉媒介要素的历时性演进

边界意识伴随着视觉艺术的成熟而来,它以种种不同的画框形式统治了视觉艺术的历史。理所当然地,物质性画框的普遍存在催生了创作中的画框意识。这种画框意识影响到了从传统绘画到先锋视觉艺术的整个历史过程,同时,也将摄影术、电影和电子游戏纳入到画框意识的话语体系中。从绘画到电子游戏,画框形式从静态演变为动态,而电子游戏则发展出了一套相当复杂且灵活的画框形式。这套画框形式丰富了画框意识的内涵,打破了传统画框结构的封闭性,呈现出了“低张力”的结构特点,而这种画框结构的形成则应归因于电子游戏的互动性本质。

人工智能技术驱动视觉传达作品生成研究综述

目的为改进人工智能技术驱动视觉传达作品的生成方式,提升视觉传达作品的生成质量并为视觉传达设计效率提供理论支撑。方法基于Scopus和中国知网数据库下载并整理相关文献,分析现有人工智能技术驱动视觉传达作品生成的关键技术、研究方向,以及研究方法。结论通过精读文献划分出目前人工智能技术驱动视觉传达作品生成的研究方向,包含以文字生成图像、以图像生成图像,以及视频生成。提取各研究方向中所采用的研究方法,涵盖生成对抗网络、知识推理、空间自适应等。通过分析人工智能技术驱动视觉作品生成的研究现状及方向,进一步总结和归纳研究方向和方法,为未来设计师应对复杂设计挑战开辟了新路径,同时为未来人工智能技术赋能视觉传达作品生成提供了参考和依据。

发展性阅读障碍儿童视觉注意广度特点及干预策略研究

为深入分析发展性阅读障碍儿童视觉注意广度缺陷,探讨视觉注意训练任务和动作视频游戏两种干预方式的效果,综合分析国内外文献,发现视觉注意广度缺陷与阅读障碍之间存在显著关联,视觉注意训练任务和动作视频游戏这两种干预方式能有效提升阅读障碍儿童的视觉注意能力和阅读表现。未来的研究应采用统一的实验范式和更标准化的材料,以提高研究结果的一致性和可比性;进一步探索阅读障碍与视觉注意广度缺陷的具体机制;进行基于不同干预方式的比较研究等。在此基础上,对未来阅读障碍儿童视觉注意广度领域的研究方向提出建议,强调进一步探索和验证的必要性。

基于语言-视觉对比学习的多模态视频行为识别方法

以对比语言-图像预训练(Contrastive language-image pre-training, CLIP)模型为基础,提出一种面向视频行为识别的多模态模型,该模型从视觉编码器的时序建模和行为类别语言描述的提示学习两个方面对CLIP模型进行拓展,可更好地学习多模态视频表达.具体地,在视觉编码器中设计虚拟帧交互模块(Virtual-frame interaction module, VIM),首先,由视频采样帧的类别分词做线性变换得到虚拟帧分词;然后,对其进行基于时序卷积和虚拟帧分词移位的时序建模操作,有效建模视频中的时空变化信息;最后,在语言分支上设计视觉强化提示模块(Visual-reinforcement prompt module,VPM),通过注意力机制融合视觉编码器末端输出的类别分词和视觉分词所带有的视觉信息来获得经过视觉信息强化的语言表达.在4个公开视频数据集上的全监督实验和2个视频数据集上的小样本、零样本实验结果,验证了该多模态模型的有效性和泛化性.

教学视频角色表征对学习影响的多模态数据融合分析

当前,教学视频已成为学生日常学习过程中不可或缺的数字学习资源,其设计与优化受到了研究者的广泛关注。但相关研究较少从教学媒体角色表征的角度出发,对不同类型的教学视频进行比较研究。为此,文章基于拟剧理论和信息技术3T论,以广东省H大学教育技术学专业的90名大学生为实验对象,设置了三组表征不同角色(教师、工具、同伴)的教学视频,通过对眼动注视数据、视频画面数据、学习成绩数据等多模态数据的融合分析,发现表征不同角色的视频对学生的学习体验和学习效果有显著影响,使用表征教师角色、同伴角色的视频学习能带来更好的学习体验和学习效果。同时,文章对不同角色表征的视频与学习效果之间的中介效应进行了分析,发现总注视时长在不同角色组教学视频与学习效果之间起中介作用,而心流体验对学习效果没有中介作用。文章的研究对于指导学生选择学习资源、优化教学设计、提升教学效果有一定的实践价值,可为数字化学习环境下的教学实践提供有益参考。

多尺度视觉特征提取及跨模态对齐的连续手语识别

连续手语识别研究中,视觉特征的有效表示是提升识别效果的关键。然而,手语动作时序长度的差异性及手语弱标注现象,使得有效的视觉特征提取更加困难。针对上述问题,提出了多尺度视觉特征提取及跨模态对齐的连续手语识别方法(MECA)。该方法主要包含多尺度视觉特征提取模型和跨模态对齐约束。在多尺度视觉特征提取模型中,并行地融合具备不同扩张因子的瓶颈残差结构,来丰富多尺度时序感受野,用于提取不同时序长度的手语视觉特征,同时采用层级复用设计进一步强化视觉特征表示。在跨模态对齐约束中,采用动态时间规整建模手语视觉特征和文本特征之间的内在联系,其中,文本特征提取由多层感知机和长短期记忆网络协作实现。在具备挑战性的公开数据集RWTH-2014、RWTH-2014T、CSL-Daily上进行实验,结果表明所提方法达到目前具有竞争力的性能。上述实验验证了所提的采用多尺度的方式可以捕捉不同时序长度的手语动作,以及构建跨模态对齐约束的思路是正确且有效的,适用于弱监督条件下的连续手语识别任务。

信息分享视域下的游戏类短视频封面设计研究

碎片化的信息流形成了人们碎片化阅读方式,游戏短视频创作者们在此背景下,为了信息分享的目的对游戏短视频封面的视觉审美进行了妥协,因而衍生出了一种特殊的视觉文化现象,那就是引导性视觉效果的滥用,这无疑会导致视觉设计和大众审美的下降。研究方法采用了文献分析法、眼动仪实验、网络收集法,对点击量较高的游戏类短视频进行研究,从信息分享理论出发,探究这种视觉文化形成的原因和合适的短视频封面设计方法。我国的短视频视觉文化还未发展成熟,虽然部分游戏类短视频封面也存在着一些低俗性,但基于信息分享理论的结果出发,考虑受众的接受心理,对受众进行适当的视觉引导,对游戏类短视频封面设计的提升有重要意义。

服装短视频可视性对购买意愿的影响研究

为了提高服装短视频视觉体验,刺激消费者的购买意愿,以抖音为切入点,基于SOR理论构建服装短视频可视性对消费者购买意愿的影响机制模型,以及感知有用性的中介作用和短视频热度的调节作用。研究表明:服装短视频可视性和消费者的感知有用性正向影响消费者购买意愿;消费者感知有用性在可视性和购买意愿中起中介作用;服装短视频热度在服装短视频可视性和消费者感知有用性之间起可调节中介作用。该研究可为服装企业提升短视频视觉营销效果提供依据,为服装短视频的营销方式和推广手段提供新思路。

融合语义信息和视觉推理特征的视频描述方法

视频描述是一项同时涉及到计算机视觉和自然语言处理两个领域的跨模态任务,其目的是为视频自动生成一段描述,所生成的内容不仅要准确完整地描述视频的主要内容,而且要符合基本的语法结构.针对现有的视频描述方法在生成过程的可解释性和生成内容的准确性等方面尚存在一些不足之处,本文提出一种基于编解码框架的融合语义信息和视觉推理特征的视频描述方法,该方法在解码阶段进行适当的改进,提出3种特征融合网络,分别为特征参与的融合网络、特征引导的融合网络以及结合权重的融合网络,将视频对应的语义特征与视觉推理特征进行融合,从而生成兼具可解释性和准确性的描述.在MSVD和MSRVTT两个数据集上进行消融和对比实验的结果表明:与基模型相比,本文所提方法的CIDEr指标分别增长了21.6%和3.5%;与其他方法的比较结果表明,本文提出的方法在各个指标上具有一定的竞争力.

移动短视频社区中的视觉人类学新特征

第五代移动通信技术的成熟将视频时代的航船推进深水区,新媒介技术的发展和变革催生了新的影像记录手段和方式,让移动视频的传播成为媒介生活的一种常态。短视频社区中的媒介生活给传统的视觉人类学学科研究拓展了新场域,自我与他者的边界消弭、“全民”影像的合力成为时代的记录与见证,视觉的“接力”形成影像叙事的传承,而技术的发展又会“赋能”学科新优势,“虚拟的田野”考察研究将成为未来人类学者必须的技能,种种变化都为视觉人类学带来新的机遇与挑战。