6-DOF motion estimation using optical flow based on dual cameras

Because of its characteristics of simple algorithm and hardware, optical flow-based motion estimation has become a hot research field, especially in GPS-denied environment. Optical flow could be used to obtain the aircraft motion information, but the six-(degree of freedom)(6-DOF) motion still couldn't be accurately estimated by existing methods. The purpose of this work is to provide a motion estimation method based on optical flow from forward and down looking cameras, which doesn't rely on the assumption of level flight. First, the distribution and decoupling method of optical flow from forward camera are utilized to get attitude. Then, the resulted angular velocities are utilized to obtain the translational optical flow of the down camera, which can eliminate the influence of rotational motion on velocity estimation. Besides, the translational motion estimation equation is simplified by establishing the relation between the depths of feature points and the aircraft altitude. Finally, simulation results show that the method presented is accurate and robust.

A Correlation Tracking Algorithm Based on Template Partition Motion Estimation

A correlation tracking algorithm based on template partition motion estimation proposed for improving real time performance of the conventional correlation matching algorithms. The target trajectory fitted using the least square with equal space in whole interval and the target prediction point is found out. According to the requirements of block motion estimation(BME) algorithm,the template divided into some macro blocks. The searching process is conducted by using diamond search algorithm around the prediction point and the optimal motion vector of each block is calculated. A point corresponding to the motion vector with the best matching is taken as a rough matching point of the template. The relation of relative position between the block with matching point and the searching area determined to decide whether to conduct precise matching search or to construct a new search area in the gradient direction. The target tracking experiment results show that over 70% time cost can be reduced caompared with the conventional correlation matching algorithm based on full search method.

Fast motion estimation algorithm for H.264/AVC based on centered prediction

H.264/AVC video coding standard can achieve roughly half of the bit-savings over MPEG2 and MPEG4 for a given quality. However, this comes at a cost in considerably increased complexity at the encoder and thus increases the difficulty in hardware implementation. The high redundancy that exists between the successive frames of a video sequence makes it possible to achieve a high data compression ratio. Motion estimation （ME） plays an important role in motion compensated video coding. A fast motion estimation algorithm for H.264/AVC is proposed based on centered prediction, called centered prediction based fast mixed search algorithm （CPFMS）. It makes use of the spatial and temporal correlation in motion vector （MV） fields and feature of all-zero blocks to accelerate the searching process. With the initialized searching point prediction, adaptive search window changing and searching direction decision, CPFMS is provided to reduce computation in block-matching process. The experimental results show that the speed of CPFMS is nearly 12 times of FS with a negligible peak signal-noise ratio （PSNR） loss. Also, the efficiency of CPFMS outperforms some popular fast algorithms such as hybrid unsymmetrical cross multi-hexagongrid search and a novel multidirectional gradient descent search evidently.

Multiresolution Video Watermarking Algorithm Exploiting the Block-Based Motion Estimation

This paper presents a novel technique for embedding a digital watermark into video frames based on motion vectors and discrete wavelet transform (DWT). In the proposed scheme, the binary image watermark is divided into blocks and each watermark block is embedded several times in each selected video frame at different locations. The block-based motion estimation algorithm is used to select the video frame blocks having the greatest motion vectors magnitude. The DWT is applied to the selected frame blocks, and then, the watermark block is hidden into these blocks by modifying the coefficients of the Horizontal sub-bands (HL). Adding the watermark at different locations in the same video frame makes the scheme more robust against different types of attacks. The method was tested on different types of videos. The average peak signal to noise ratio (PSNR) and the normalized correlation (NC) are used to measure the performance of the proposed method. Experimental results show that the proposed algorithm does not affect the visual quality of video frames and the scheme is robust against a variety of attacks.

Fast Approaches for Sub-pixel Precision Variable Block Size Motion Estimation

The new features of H. 264 video coding standard make the motion estimation module much more time consuming than before. Especially, the motion search is required for each of the 4 modes for inter prediction. In order to reduce the computational complexity, we analyze the statistics of results of motion estimation, such as the continuity of best modes of blocks in successive frames and the chance to give up a sub-partition mode （smaller than 16 × 16） after integer-pixel motion estimation, from which we suggest to make mode prediction based on the motion information of the previous frame and skip sub-pixel motion estimation in subpartition mode selectively. According to the experimental result, the proposed algorithm can save 75 % of the computational time with a slight degradation （0.03 dB） on PSNR compared with the pseudocode of fast search motion estimation in JM12.2.

Fast motion estimation algorithm of video stabilization system

A real-time electronic image stabilization motion estimation method based on fast sub- block gray projection algorithm is proposed. In the method, each image is divided into a number of sub-blocks, and sub-blocks are sifted with their gray gradients. After removing sub-blocks whose gray gradients are lower than the given threshold, the calculation amount of projection is reduced and the motion estimation accuracy is improved. Then gray projection is done in each remained sub- block, and global motion vector of the image is calculated according to the local motion vectors of sub-blocks and the affine motion model. The drawbacks as the local motions reducing the global mo- tion estimation accuracy and traditional gray projection algorithm could not deal with rotation are re- solved well by this algorithm. The experiment results show that the algorithm is more accurate and efficient than the gray projection algorithm.

Global motion estimation with Gabor wavelet transform

A novel algorithm of global motion estimation is proposed. First, through Gabor wavelet transform （GWT）, a kind of energy distribution of image is obtained and checkpoints are selected according to a probability decision approach proposed. Then, the initialized motion vectors are obtained via a hierarcbal block-matching based on these checkpoints. Finally, by employing a 3-parameter motion model, precise parameters of global motion are found. From the experiment, the algorithm is reliable and robust.

Reduced bit low power VLSI architectures for motion estimation

Low power and real time very large scale integration （VLSI） architectures of motion estimation （ME） algorithms for mobile devices and applications are presented. The power reduction is achieved by devising a novel correction recovery mechanism based on algorithms which allow the use of reduced bit sum of absolute difference （RBSAD） metric for calculating matching error and conversion to full resolution sum of absolute difference （SAD） metric whenever necessary. Parallel and pipelined architectures for high throughput of full search ME corresponding to both the full resolution SAD and the generalized RBSAD algorithm are synthe- sized using Xilinx Synthesis Tools （XST）, where the ME designs based on reduced bit （RB） algorithms demonstrate the reduction in power consumption up to 45% and/or the reduction in area up to 38%.

NOVEL TWO-LAYER MOTION ESTIMATION FOR VIDEO CODING

In this paper, we proposed a novel Two-layer Motion Estimation(TME) which searches motion vectors on two layers with partial distortion measures in order to reduce the overwhelming computational complexity of Motion Estimation(ME) in video coding. A layer is an image which is derived from the reference frame such that the sum of a block of pixels in the reference frame determines the point of a layer. It has been noticed on different video sequences that many motion vectors on the layers are the same as those searched on the reference frame. The proposed TME performs a coarse search on the first layer to identify the small region in which the best candidate block is likely to be positioned and then perform local refined search on the next layer to pick the best candidate block in the located small area. The key feature of TME is its flexibility of mixing with any fast search algorithm. Experimental results on a wide variety of video sequences show that the proposed algorithm has achieved both fast speed and good motion prediction quality when compared to well known as well as the state-of-the-art fast block matching algorithms.

Advanced 4-dimensional cone-beam computed tomography reconstruction by combining motion estimation, motioncompensated reconstruction, biomechanical modeling and deep learning

4-Dimensional cone-beam computed tomography(4D-CBCT)offers several key advantages over conventional 3DCBCT in moving target localization/delineation,structure de-blurring,target motion tracking,treatment dose accumulation and adaptive radiation therapy.However,the use of the 4D-CBCT in current radiation therapy practices has been limited,mostly due to its sub-optimal image quality from limited angular sampling of conebeam projections.In this study,we summarized the recent developments of 4D-CBCT reconstruction techniques for image quality improvement,and introduced our developments of a new 4D-CBCT reconstruction technique which features simultaneous motion estimation and image reconstruction(SMEIR).Based on the original SMEIR scheme,biomechanical modeling-guided SMEIR(SMEIR-Bio)was introduced to further improve the reconstruction accuracy of fine details in lung 4D-CBCTs.To improve the efficiency of reconstruction,we recently developed a U-net-based deformation-vector-field(DVF)optimization technique to leverage a population-based deep learning scheme to improve the accuracy of intra-lung DVFs(SMEIR-Unet),without explicit biomechanical modeling.Details of each of the SMEIR,SMEIR-Bio and SMEIR-Unet techniques were included in this study,along with the corresponding results comparing the reconstruction accuracy in terms of CBCT images and the DVFs.We also discussed the application prospects of the SMEIR-type techniques in image-guided radiation therapy and adaptive radiation therapy,and presented potential schemes on future developments to achieve faster and more accurate 4D-CBCT imaging.

C-Testable Motion Estimation Design for Video Coding Systems

A C-testable motion estimation （CTME） design to efficiently detect the faults in process elements （PEs） is presented. The goal of the CTME design is to offer high reliability for video coding systems. The proposed CTME was carried out by Verilog HDL （very high speed integrated circuit hardware description language） and synthesized with the TSMC （Taiwan Semiconductor Manufacturing Company） 0.18 μm CMOS technology. Logic simulation results show that the proposed CTME guarantees 100% fault coverage with only 8 sets of test patterns. Design verification and comparisons also demonstrate that the area overhead of the proposed CTME is about 4.8% with little delay penalty.

Bioinspired Polarized Optical Flow Enables Turbid Underwater Target Motion Estimation

Underwater target motion estimation is a challenge for ocean military and scientific research.In this work,we propose a method based on the combination of polarization imaging and optical flow for turbid underwater target detection.Polarization imaging can reduce the influence of backscattered light and obtain high-quality images underwater.The optical flow shows the motion and structural information of the target.We use polarized optical flow to obtain the optical flow field and estimate the target motion.The experimental results of different targets under varying water turbidity levels illustrate that our method is realizable and robust.The precision is verified by comparing the results with the precise displacement data and calculating two error measures.The proposed method based on polarized optical flow can obtain accurate displacement information and a good recognition effect.Moving target segmentation based on the Otsu method further proves the superiority of the polarized optical flow under turbid water.This study is valuable for target detection and motion estimation in scattering environments.

6-DOF MOTION AND CENTER OF ROTATION ESTIMATION BASED ON STEREO VISION

A new motion model and estimation algorithm is proposed to compute the general rigid motion object＇s 6-DOF motion parameters and center of rotation based on stereo vision. The object＇s 6-DOF motion model is designed from the rigid object＇s motion character under the two defined reference frames. According to the rigid object＇s motion model and motion dynamics knowledge, the corresponding motion algorithm to compute the 6-DOF motion parameters is worked out. By the rigid object pure rotation motion model and space sphere geometry knowledge, the center of rotation may be calculated after eliminating the translation motion out of the 6-DOF motion. The motion equations are educed based on the motion model and the closed-form solutions are figured out. To heighten the motion estimation algorithm＇s robust, RANSAC algorithm is applied to delete the outliers. Simulation and real experiments are conducted and the experiment results are analyzed. The results prove the motion model＇s correction and algorithm＇s validity.

Adaptive Noise Identification in Vision-assisted Motion Estimation for Unmanned Aerial Vehicles

Vision localization methods have been widely used in the motion estimation of unmanned aerial vehicles(UAVs).The noise of the vision location result is usually modeled as a white Gaussian noise so that this location result could be utilized as the observation vector in the Kalman filter to estimate the motion of the vehicle.Since the noise of the vision location result is affected by external environment,the variance of the noise is uncertain.However,in previous researches,the variance is usually set as a fixed empirical value,which will lower the accuracy of the motion estimation.The main contribution of this paper is that we proposed a novel adaptive noise variance identification(ANVI) method,which utilizes the special kinematic properties of the UAV for frequency analysis and then adaptively identifies the variance of the noise.The adaptively identified variance is used in the Kalman filter for more accurate motion estimation.The performance of the proposed method is assessed by simulations and field experiments on a quadrotor system.The results illustrate the effectiveness of the method.

Wavelets and Optical Flow Motion Estimation

This article describes the implementation of a simple wavelet-based optical-flow motion estimator dedicated to continuous motions such as fluid flows. The waveletrepresentation of the unknown velocity field is considered. This scale-space representation, associated to a simple gradient-based optimization algorithm, sets up a welldefined multiresolution framework for the optical flow estimation. Moreover, a verysimple closure mechanism, approaching locally the solution by high-order polynomialsis provided by truncating the wavelet basis at fine scales. Accuracy and efficiency of theproposed method is evaluated on image sequences of turbulent fluid flows.

A Content Based Search Algorithm for Motion Estimation

The basic search algorithm to implement Motion Estimation ( ME ) in the H .263 encoder is a full search. It is simple but time consuming. Traditional search algorithms are fast, but may cause a fall in image quality or an increase in bit rate in low bit rate applications. A fast search algorithm for ME with consideration on image content is proposed in this paper. Experiments show that the proposed algorithm can offer up to 70 percent savings in execution time with almost no sacrifice in PSNR and bit rate, compared with the full search.

A pixel-based outlier-free motion estimation algorithm for scalable video quality enhancement

Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always in- troduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scale- invariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.

Multi-Exposure Motion Estimation Based on Deep Convolutional Networks

In motion estimation, illumination change is always a troublesome obstacle, which often causes severely per- formance reduction of optical flow computation. The essential reason is that most of estimation methods fail to formalize a unified definition in color or gradient domain for diverse environmental changes. In this paper, we propose a new solution based on deep convolutional networks to solve the key issue. Our idea is to train deep convolutional networks to represent the complex motion features under illumination change, and further predict the final optical flow fields. To this end, we construct a training dataset of multi-exposure image pairs by performing a series of non-linear adjustments in the traditional datasets of optical.flow estimation. Our multi-exposure flow networks （MEFNet） model consists of three main components： low-level feature network, fusion feature network, and motion estimation network. The former two components belong to the contracting part of our model in order to extract and represent the multi-exposure motion features; the third component is the expanding part of our model in order to learn and predict the high-quality optical flow. Compared with many state- of-the-art methods, our motion estimation method can eliminate the obstacle of illumination change and yield optical flow results with competitive accuracy and time efficiency. Moreover, the good performance of our model is also demonstrated in some multi-exposure video applications, like HDR （high dynamic range） composition and flicker removal.

A Modified Block Matching Motion Estimation Algorithm for the Very Low Bit-Rate video Coding

Block matching motion estimation techniques have been widely used in video coding applications. However, they also show their deficiency in the coherence of motion vectors and antinoise ability. This paper proposes a modified algorithm which can adopt any one of existing search algorithms and pays more attention to the correlation of neighboring blocks.It will be shown that the proposed algorithm is simple and significantly reduces the computational complexity. Simulation results also show that this algorithm improves the smoothness of the motion field, hence reducing the cost to code the motion vectors while keeping good performance comparable with the conventional block matching motion estimation algorithm.

Super-Resolution Reconstruction of Image Sequence Using Multiple Motion Estimation Fusion

Super-resolution reconstruction algorithm produces a high-resolution imagefrom a low-resolution image sequence. The accuracy and the stability of the motion estimation (ME)are essential for the whole restoration. In this paper, a new super-resolution reconstructionalgorithm is developed using a robust ME method, which fuses multiple estimated motion vectorswithin the sequence. The new algorithm has two major improvements compared with the previousresearch. First, instead of only two frames, the whole sequence is used to obtain a more accurateand stable estimation of the motion vector of each frame; second, the reliability of the ME isquantitatively measured and introduced into the cost function of the reconstruction algorithm. Thealgorithm is applied to both synthetic and real sequences, and the results are presented in thepaper.