基于FPGA的H.264可变块运动估计算法的优化及实现

可变块运动估计算法(Variable Block Size Motion Estimation,VBSME)是H.264标准中重要的组成部分。它不仅计算量大,而且耗时长。为了减少运动估计时间和计算量,本文采用硬件实现的方式,并提出了一种利用绝对差值和(Sum of Absolute Differences,SAD)计算的树状结构进行数据重用的方案。该方案使得各单元间的数据流向明确,结构更简单。同时,本文还考虑到了帧间模式决策(Mode Decision,MD)和SAD计算可以并行计算的可行性,设计了相应的并行流水线结构。利用Xilinx xc7v585tffg1761-1开发板进行了仿真验证。结果表明,该方案可以一次性处理输入的256个像素数据,提高了实时性,并且达到了100%的数据利用率。此外,该方案支持最大分辨率为1920×1080,帧率为60帧/s,并具备低编码延时,满足了绝大多数场合的实时性要求。

基于申威SIMD指令的H.264编码优化

国产化申威处理器出现较晚,其在多媒体领域中的性能还不突出,同时通用处理器中的单指令流多数据流(SIMD)因能有效提升并行处理能力而受到处理器厂商的青睐。为提高国产化自主平台申威架构的多媒体处理能力,结合申威架构Core3B体系的SIMD指令系统,提出一种基于申威架构的SIMD指令集H.264编码优化方法。结合申威处理器的并行结构特点,利用申威适配的Perf、Top指令等系统性能分析工具,采集两种主流视频分辨率下与编码性能强相关的高频热点函数,详细分析其程序并行化可行性,采用手工嵌入申威SIMD和访存扩展等汇编指令进行细粒度优化。实验结果表明,该方法在申威架构下的H.264平均编码性能提升了约30%。相应工作成果已推送到申威社区,增强了基于申威处理器的国产计算机在桌面多媒体应用领域的工作体验。

基于威焱831平台的H.264视频解码优化

为提高威焱831平台的多媒体处理能力,解决H.264解码器解码效率低的问题,在提出SIMD指令级优化方法的同时,提出一种面向帧拷贝的优化方法。通过分析开源软件FFmpeg中H.264解码器的并行化特性,使用威焱平台性能分析工具解析影响视频解码性能的热点函数。采用手工嵌入SIMD汇编指令的方式对关键模块热点函数进行优化,通过FFmpeg源码编译过程链接汇编实现的内存操作函数memcpy提升内存拷贝速度。实验结果表明,威焱831平台视频解码的平均性能提高26%,推动了威焱831处理器在多媒体应用领域的发展。

新一代通用视频编码标准H.266/VVC:现状与发展

相比于上一代标准,新一代通用视频编码标准(H.266/VVC)在同等质量下能够节省大约50%的码率,且适用于多种多样的视频应用场景。论文从H.266/VVC的关键技术出发,对标准的现状、实现和应用发展进行深入探讨。H.266/VVC沿用既往标准中的双层码流体系和混合编码框架,针对帧内预测、帧间预测、变换、量化、环路滤波等所有主要编码模块进行了技术革新,并为屏幕内容视频等应用提供了高效的专用编码工具。H.266/VVC标准目前已处于实用化阶段,官方参考软件VTM和开源编解码器VVenC/VVdeC是目前最具代表性的软件编解码实现。对H.266/VVC的性能分析可以看出:H.266/VVC针对高分辨率视频取得的编码增益更为突出;主要编码工具对性能的贡献通常以复杂度为代价,但也有部分编码工具在提升编码性能的同时可降低整体编码复杂度。H.266/VVC的硬件实现面临诸多挑战,发展明显滞后于软件实现,现有研究主要集中在对具体编码模块的硬件加速方面。H.266/VVC标准发布之后,下一代视频编码标准的发展目前仍围绕混合编码框架进行探索,聚焦在两大方向:超越VVC的增强压缩关注更为先进的、非神经网络的编码工具,基于神经网络的视频编码则探索采用神经网络的编码工具。除此之外,部分或完全跳出现有混合编码框架的端到端视频编码也在飞速发展,未来视频编码标准与神经网络结合成为趋势,但面临着计算资源依赖和稳定结构两方面的考验。

H.264/AVC自适应视频水印算法

H.264/AVC视频数字水印在视频版权保护和信息隐藏等方面具有重要作用。目前,已有的水印研究,在水印嵌入过程中缺乏考虑视频画面复杂程度不断变化的问题,从而易导致两种后果:一是可能无法在画面复杂程度低的帧完整嵌入所需水印;二是无法充分利用画面复杂程度高的帧的冗余度。由此,分析在视频编码过程中与视频画面复杂程度相关的因素,提出一种根据视频画面复杂程度自适应嵌入水印的视频数字水印算法。在所提算法中,根据上一关键帧中符合嵌入条件的Intra_4×4子宏块的总个数预测获得当前关键帧的画面复杂度,并依据此复杂度自动调整每一水印比特重复嵌入的次数,从而达到自适应的目的。当由于预测失误而导致某一帧的水印嵌入失败,水印提取端则会舍弃该帧提取出的水印。提取端采用多数表决的机制提取水印。实验结果显示,该算法的水印具有良好的不可见性,且能保证码流的稳定性,优于近期相关文献的算法。算法对以重复嵌入水印比特为基础的研究具有启发意义。

浅谈H.264 H.265视频编码及其在安防系统的应用

面对视频应用向高画质、高帧率、高压缩比方向发展的趋势,视频编解码标准也在日益更新,阐述了主要视频压缩标准H.264和H.265的主要技术参数和各自的优缺点,重点分析了H.265的关键技术及核心优势,探讨了H.265与安防系统结合给行业带来的新机遇。

RTP payload format for H.264/SVC scalable video coding

The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC. Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL) and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, in- cluding the design criteria, use cases, signaling and payload structure.

An adaptive joint source and channel coding scheme for H.264 video multicasting over wireless LAN

This paper proposes an adaptive joint source and channel coding scheme for H.264 video multicast over wireless LAN which takes into account the user topology changes and varying channel conditions of multiple users, and dynamically allocates available bandwidth between source coding and channel coding, with the goal to optimize the overall system performance. In particular, source resilience and error correction are considered jointly in the scheme to achieve the optimal performance. And a channel estimation algorithm based on the average packet loss rate and the variance of packet loss rate is proposed also. Two overall performance criteria for video multicast are investigated and experimental results are presented to show the improvement obtained by the scheme.

Fast mode decision algorithm for spatial resolutions down-scaling transcoding to H.264

A fast mode decision algorithm is proposed in this paper to accelerate the process of transcoding videos into H.264with arbitrary rate spatial resolution down-scaling. The proposed algorithm consists of three steps. First, an early-stop technique is introduced to determine the 16× 16-mode blocks, which take up about 70% of all the macroblocks; then, a bottom-up merging process is performed to determine the mode of rest non-early-stopped blocks; and then, we adopt half-pixel motion estimation to further refine the acquired predictive motion vectors. In order to obtain the predictive motion vectors for early-stop and merging processes, we propose a motion vector composition scheme, which can reuse the information in the input pre-encoded videos to handle the spatial resolution down-scaling. Experimental results showed that our algorithm is about four times faster than the Cascaded-Decoder-Encoder method and has negligible PSNR drop and little bit rate increase.

An error resilient scheme for H.264 video coding based on distortion estimated mode decision and nearest neighbor error concealment

Although H.264 video coding standard provides several error resilience tools, the damage caused by error propagation may still be tremendous. This work is aimed at developing a robust and standard-compliant error resilient coding scheme for H.264and uses techniques of mode decision, data hiding, and error concealment to reduce the damage from error propagation. This paper proposes a system with two error resilience techniques that can improve the robustness of H.264 in noisy channels. The first technique is Nearest Neighbor motion compensated Error Concealment (NNEC) that chooses the nearest neighbors in the reference frames for error concealment. The second technique is Distortion Estimated Mode Decision (DEMD) that selects an optimal mode based on stochastically distorted frames. Observed simulation results showed that the rate-distortion performances of the proposed algorithms are better than those of the compared algorithms.

Resynchronization and remultiplexing for transcoding to H.264/AVC

H.264/MPEG-4 AVC standard appears highly competitive due to its high efficiency, flexibility and error resilience. In order to maintain universal multimedia access, statistical multiplexing, or adaptive video content delivery, etc., it induces an immense demand for converting a large volume of existing multimedia content from other formats into the H.264/AVC format and vice versa. In this work, we study the remultiplexing and resynchronization issue within system coding after transcoding, aiming to sustain the management and time information destroyed in transcoding and enable synchronized decoding of decoder buffers over a wide range of retrieval or receipt conditions. Given the common intention of multiplexing and synchronization mechanism in system coding of different standards, this paper takes the most widely used MPEG-2 transport stream (TS) as an example, and presents a software system and the key technologies to solve the time stamp mapping and relevant buffer management. The solution reuses previous information contained in the input streams to remultiplex and resynchronize the output information with the regulatory coding and composition structure. Experimental results showed that our solutions efficiently preserve the performance in multimedia presentation.

New Fragile Watermarking Technique to Identify Inserted Video Objects Using H.264 and Color Features

The transmission of video content over a network raises various issues relating to copyright authenticity,ethics,legality,and privacy.The protection of copyrighted video content is a significant issue in the video industry,and it is essential to find effective solutions to prevent tampering and modification of digital video content during its transmission through digital media.However,there are stillmany unresolved challenges.This paper aims to address those challenges by proposing a new technique for detectingmoving objects in digital videos,which can help prove the credibility of video content by detecting any fake objects inserted by hackers.The proposed technique involves using two methods,the H.264 and the extraction color features methods,to embed and extract watermarks in video frames.The study tested the performance of the system against various attacks and found it to be robust.The evaluation was done using different metrics such as Peak-Signal-to-Noise Ratio(PSNR),Mean Squared Error(MSE),Structural Similarity Index Measure(SSIM),Bit Correction Ratio(BCR),and Normalized Correlation.The accuracy of identifying moving objects was high,ranging from 96.3%to 98.7%.The system was also able to embed a fragile watermark with a success rate of over 93.65%and had an average capacity of hiding of 78.67.The reconstructed video frames had high quality with a PSNR of at least 65.45 dB and SSIMof over 0.97,making them imperceptible to the human eye.The system also had an acceptable average time difference(T=1.227/s)compared with other state-of-the-art methods.

Fast Inter Coding Mode Selection Algorithm for H.264

In order to reduce the encoding complexity of macroblock coding mode decision in H.264/AVC, a selective smaller block-size searching algorithm and a selective intra coding mode searching algorithm are proposed by using the high correlation among coding modes and in spatial and temporal domains of video sequence. Simulation results demonstrate that the proposed algorithm can provide significant improvement in computational requirement, with negligible small picture quality degradation and slight bit rate increase.

Integrated Joint Source-Channel Decoding for H.264 Video Transmission

This paper proposes an integrated joint source-channel decoder （I-JSCD） using Max-Log-MAP method for sources encoded with exp-Golomb codes and convolutional codes, and proposes a system applying this method to decoding the VLC data, e.g. motion vector differences （MVDs）, of H.264 across an AWGN channel. This method combines the source code state-space and the channel code state-space together to construct a joint state-space, develops a 3-D trellis and a maximum a-posterior （MAP） algorithm to estimate the source sequence symbol by symbol, and then uses max-log approximation to simplify the algorithm. Experiments indicate that the proposed system gives significant improvements on peak signal-to-noise ratio （PSNR） （maximum about 15 dB） than a separate scheme. This also leads to a higher visual quality of video stream over a highly noisy channel.

A Dual Domain Robust Reversible Watermarking Algorithm for Frame Grouping Videos Using Scene Smoothness

The proposed robust reversible watermarking algorithm addresses the compatibility challenges between robustness and reversibility in existing video watermarking techniques by leveraging scene smoothness for frame grouping videos.Grounded in the H.264 video coding standard,the algorithm first employs traditional robust watermark stitching technology to embed watermark information in the low-frequency coefficient domain of the U channel.Subsequently,it utilizes histogram migration techniques in the high-frequency coefficient domain of the U channel to embed auxiliary information,enabling successful watermark extraction and lossless recovery of the original video content.Experimental results demonstrate the algorithm’s strong imperceptibility,with each embedded frame in the experimental videos achieving a mean peak signal-to-noise ratio of 49.3830 dB and a mean structural similarity of 0.9996.Compared with the three comparison algorithms,the performance of the two experimental indexes is improved by 7.59%and 0.4%on average.At the same time,the proposed algorithm has strong robustness to both offline and online attacks:In the face of offline attacks,the average normalized correlation coefficient between the extracted watermark and the original watermark is 0.9989,and the average bit error rate is 0.0089.In the face of online attacks,the normalized correlation coefficient between the extracted watermark and the original watermark is 0.8840,and the mean bit error rate is 0.2269.Compared with the three comparison algorithms,the performance of the two experimental indexes is improved by 1.27%and 18.16%on average,highlighting the algorithm’s robustness.Furthermore,the algorithm exhibits low computational complexity,with the mean encoding and the mean decoding time differentials during experimental video processing being 3.934 and 2.273 s,respectively,underscoring its practical utility.

Improved Diamond Search Algorithm for H.264/AVC Video Coding Standard

Diamond search （DS） is an excellent fast block matching motion estimation （BMME） algorithm. In this paper, we propose an improved diamond search （IDS） algorithm, which revises the two search patterns of DS. The proposed algorithm is compared with several mainstream algorithms. The simulation results show that the proposed algorithm over DS can be up to 20% gain on speedup on average, while maintain the similar or even better quality, both objectively and subjectively. The proposed algorithm is also competitive with other fast algorithms.

Complexity scalable control algorithm for intra coding in H.264 under energy constraints

In this paper, the complexity of intra coding is first analyzed so as to achieve a weight of complexity measurement for each intra mode. Then, a new complexity scalable control algorithm for intra coding in H. 264 is proposed, based on the rearrangement of the order of candidate modes and an efficient complexity allocation and control （CAAC） scheme at the macroblock （MB） level. The candidate modes of each MB are rearranged according to the local-edge information. Experimental results show that our proposed algorithm can make an appropriate cut-off point of the candidate modes sequence adaptively according to the current energy condition of a mobile device, so as to adjust the complexity at any level while maximizing the video quality, which can prolong the operational lifetime of the battery with minimum degradation in video quality.

Frame complexity optimized selection for H.264/AVC video coding

To improve the coding performance of H.264/AVC, this paper proposes a rate control scheme composed of a novel flame complexity optimized selection and a quantization parameter （QP） value computation approach. First, it extracts the frame coding complexity from two rate distortion models, and then introduces five statistic modes to estimate the frame coding complexity. An optimal mode is selected according to the coding efficiency. Finally the paper presents a novel QP calculation method for the H.264/AVC rate control. Experimental results show that the proposed algorithra outperforms the algorithm integrated in the 3M model in obtaining precise frame coding complexity, achieving robust buffer control and improving coding quality. And the improving visual quality is high up to 0.90dB for CIF sequences.

Mode decision for H.264 down-scaling video transcoding

Although the coding modes of H.264 coded video would be changed by the transcoding process of spatial resolution reduction, there exists good correlation in prediction modes and prediction directions between input and output video. In this paper, we first introduce a new spatial resolution reduction transcoding architecture of intra coded frames where the distortion can be calculated directly in compression domain. We then propose a fast mode decision algorithm in which only a small part of rate distortion optimization （RDO） calculation is needed for mode decision. For 4×4 luma block, the proposed scheme has average 21.3% computation saving, compared to the cascaded pixel-domain transcoding scheme with the fast intra mode decision algorithm proposed in JVT-G013. For 16×16 luma block, RDO calculation is completely avoided in our scheme while the scheme in JVT-G013 needs 2 RDO calculations. Experimental results show that our scheme outperforms that of JVT-G013 in terms of significantly computasavings with negligible loss of PSNR

Reversible Data Hiding Based on Run-Level Coding in H.264/AVC Video Streams

This paper presents a reversible data hiding(RDH)method,which is designed by combining histogram modification(HM)with run-level coding in H.264/advanced video coding(AVC).In this scheme,the run-level is changed for embedding data into H.264/AVC video sequences.In order to guarantee the reversibility of the proposed scheme,the last nonzero quantized discrete cosine transform(DCT)coefficients in embeddable 4×4 blocks are shifted by the technology of histogram modification.The proposed scheme is realized after quantization and before entropy coding of H.264/AVC compression standard.Therefore,the embedded information can be correctly extracted at the decoding side.Peak-signal-noise-to-ratio(PSNR)and Structure similarity index(SSIM),embedding payload and bit-rate variation are exploited to measure the performance of the proposed scheme.Experimental results have shown that the proposed scheme leads to less SSIM variation and bit-rate increase.