Frame interpolation with pixel-level motion vector field and mesh based hole filling

Most of the traditional methods are based on block motion compensation tending to involve heavy blocking artifacts in the interpolated frames. In this paper, a new frame interpolation method with pixel-level motion vector field （MVF） is proposed. Our method consists of the following four steps： （i） applying the pixel-level motion vectors （MVs） estimated by optical flow algorithm to eliminate blocking artifacts （ii） motion post-processing and super-sampling anti-aliasing to solve the problems caused by pixel-level MVs （iii） robust warping method to address collisions and holes caused by occlusions （iv） a new holes filling method using triangular mesh （HFTM） to reduce the artifacts caused by holes. Experimental results show that the proposed method can effectively alleviate the holes and blocking artifacts in interpolated frames, and outperforms existing methods both in terms of objective and subjective performances, especially for sequences with complex motions.

Flow-aware synthesis: A generic motion model for video frame interpolation

A popular and challenging task in video research,frame interpolation aims to increase the frame rate of video.Most existing methods employ a fixed motion model,e.g.,linear,quadratic,or cubic,to estimate the intermediate warping field.However,such fixed motion models cannot well represent the complicated non-linear motions in the real world or rendered animations.Instead,we present an adaptive flow prediction module to better approximate the complex motions in video.Furthermore,interpolating just one intermediate frame between consecutive input frames may be insufficient for complicated non-linear motions.To enable multi-frame interpolation,we introduce the time as a control variable when interpolating frames between original ones in our generic adaptive flow prediction module.Qualitative and quantitative experimental results show that our method can produce high-quality results and outperforms the existing stateof-the-art methods on popular public datasets.

Moving Human Posture Recognition Based on Joint Quaternion

Posture recognition plays an important role in many applications,such as security system and monitoring system.Joint quaternion combined with support vector machine(SVM) can solve the problem of moving human posture recognition.It is a simple and effective algorithm that only three joints are used as the feature points in the whole human skeleton.Using the quaternion of the three joints,a feature vector with five parameters in gait cycle is extracted.The efficiency of the proposed method is demonstrated through an experimental study,and walking and running postures can be distinguished accurately.

Study of frame-rate up conversion based on H.264

In this study, a low complexity frame-rate up conversion method using compressed domain information for H.264 decoder is proposed. In the proposed scheme, the motion vectors （MVs） are estimated using constant acceleration motion model, and the MVs regarded as no credibility are corrected, and the interpolation method is applied on the basis of the macroblock （MB） coded types. Applied to the H.264 decoder, the proposed method provides high quality interpolation frames and an obvious decrease of the block artifacts.

Side Information Generation Based on Hierarchical Motion Estimation in Distributed Video Coding

The side information quality has an immense effect on the compression efficiency of the distributed video coding (DVC) sys- tem. This article, based on the hierarchical motion estimation (HME), proposes a new side information generation algorithm which is integrated into DVC system. First, forward motion estimation (FME) and bidirectional motion estimation (BME) on the basis of variable block size HME algorithm are used to acquire relatively accurate motion vectors. Second, a motion vector filter (MVF) is i...

Time analysis of regional structure of large-scale particle using an interactive visual system

N-body numerical simulation is an important tool in astronomy.Scientists used this method to simulate the formation of structure of the universe,which is key to understanding how the universe formed.As research on this subject further develops,astronomers require a more precise method that enables expansion of the simulation and an increase in the number of simulation particles.However,retaining all temporal information is infeasible due to a lack of computer storage.In the circumstances,astronomers reserve temporal data at intervals,merging rough and baffling animations of universal evolution.In this study,we propose a deep-learning-assisted interpolation application to analyze the structure formation of the universe.First,we evaluate the feasibility of applying interpolation to generate an animation of the universal evolution through an experiment.Then,we demonstrate the superiority of deep convolutional neural network(DCNN)method by comparing its quality and performance with the actual results together with the results generated by other popular interpolation algorithms.In addition,we present PRSVis,an interactive visual analytics system that supports global volume rendering,local area magnification,and temporal animation generation.PRSVis allows users to visualize a global volume rendering,interactively select one cubic region from the rendering and intelligently produce a time-series animation of the high-resolution region using the deep-learning-assisted method.In summary,we propose an interactive visual system,integrated with the DCNN interpolation method that is validated through experiments,to help scientists easily understand the evolution of the particle region structure.

Temporally resolving premixed turbulent flame structures using self-supervised adversarial reconstruction of CH-PLIF

Understanding the turbulence-flame interaction is crucial to model the low-emission combustors developed for energy and propulsion applications. To this end, a novel frame interpolation (FI) method is proposed to better resolve the spatiotemporal evolution of premixed turbulent flame structures. The framework is completely selfsupervised, agnostic to optical flow, and driven by leveraging transferrable feature knowledge at lower speeds and adversarial learning to statistically map the flame dynamics across frames. The method is successfully applied on a 10 kHz CH planar laser-induced fluorescence (PLIF) dataset of highly wrinkled premixed flames with turbulent Reynolds numbers (ReT ) of 1100, 1400, and 7900, by down-sampling the image sequence to 5 kHz and restoring the sequence back to 10 kHz via FI. All reconstructions recovered important flame events and displayed excellent resemblance of the corrugated CH-layer geometries to that of the ground truths, with average intersection over union (IoU) and structural similarity index (SSIM) scores of 0.49 and 0.82, which are above the high-similarity baselines of 0.36 and 0.75, respectively. The wrinkling parameters (WP) of the flames also matched the ground truths, wherein R2 was roughly 0.95 for ReT = 1100 and 1400 and 0.85 for ReT = 7900 (lower due to the turbulence-induced uncertainties). The FI is further iteratively repeated to 40 kHz on the ReT = 7900 flames to facilitate pocket analysis by confidently linking their origin of formation, thus, enabling distinction from 3D tunnels, and improving statistical characterization of their consumption speeds. Given that the object features do not exhibit highly turbulent motions with regard to the initial time step, the proposed FI method is shown to be highly accurate and useful to analyzing finite-resolution experimental image sets including, but not restricted to, CH-PLIF.