A Content-Aware Bitrate Selection Method Using Multi-Step Prediction for 360-Degree Video Streaming

A content-aware multi-step prediction control(CAMPC)algorithm is proposed to determine the bitrate of 360-degree videos,aim⁃ing to enhance the quality of experience(QoE)of users and reduce the cost of video content providers(VCP).The CAMPC algorithm first em⁃ploys a neural network to generate the content richness and combines it with the current field of view(FOV)to accurately predict the probability distribution of tiles being viewed.Then,for the tiles in the predicted viewport which directly affect QoE,the CAMPC algorithm utilizes a multi-step prediction for future system states,and accordingly selects the bitrates of multiple subsequent steps,instead of an instantaneous state.Meanwhile,it controls the buffer occupancy to eliminate the impact of prediction errors.We implement CAMPC on players by building a 360-degree video streaming platform and evaluating other advanced adaptive bitrate(ABR)rules through the real network.Experimental results show that CAMPC can save 83.5%of bandwidth resources compared with the scheme that completely transmits the tiles outside the viewport with the Dynamic Adaptive Streaming over HTTP(DASH)protocol.Besides,the proposed method can improve the system utility by 62.7%and 27.6%compared with the DASH official and viewport-based rules,respectively.

Optimized Content-Aware Image Resizing with Merging and Improved Importance Diffusion

A simple and effective content-aware image resizing method is proposed based on the row / column merging and improved importance diffusion,which preserves the important regions in an image as well as the global visual effect. By repeatedly merging two rows / columns into one row / column or inserting a new row /column between two rows / columns, this method realizes image-resolution reduction and expansion. The importance of the merged row / column is promoted and diffused to four rows / columns around the merged one,which is to avoid the unwanted image distortions resulted from excessively merging of un-important regions. In addition,the proposed method introduces the direction of gradient vector in the low-pass filter to reduce the interference caused by complex texture background and protect important content better. Furthermore,according to human mechanics principles,different weights are given to the row and column direction components of gradient vectors which can obtain better global visual effect. Experimented results show that the proposed method satisfied in not only visual effect but also objective evaluation.

浅析如何对视频内容进行识别填充

本文探讨用AfterEffects的Content-Aware以及Mocha软件对视频内容进行擦除。

Visual salience guided feature-aware shape simplification

In the area of 3D digital engineering and 3D digital geometry processing, shape simplification is an important task to reduce their requirement of large memory and high time complexity. By incorporating the content-aware visual salience measure of a polygonal mesh into simplification operation, a novel feature-aware shape simplification approach is presented in this paper. Owing to the robust extraction of relief heights on 3D highly detailed meshes, our visual salience measure is defined by a center-surround operator on Gaussian-weighted relief heights in a scale-dependent manner. Guided by our visual salience map, the feature-aware shape simplification algorithm can be performed by weighting the high-dimensional feature space quadric error metric of vertex pair contractions with the weight map derived from our visual salience map. The weighted quadric error metric is calculated in a six-dimensional feature space by combining the position and normal information of mesh vertices. Experimental results demonstrate that our visual salience guided shape simplification scheme can adaptively and effectively re-sample the underlying models in a feature-aware manner, which can account for the visually salient features of the complex shapes and thus yield better visual fidelity.