Bioinspired Polarized Optical Flow Enables Turbid Underwater Target Motion Estimation

Underwater target motion estimation is a challenge for ocean military and scientific research.In this work,we propose a method based on the combination of polarization imaging and optical flow for turbid underwater target detection.Polarization imaging can reduce the influence of backscattered light and obtain high-quality images underwater.The optical flow shows the motion and structural information of the target.We use polarized optical flow to obtain the optical flow field and estimate the target motion.The experimental results of different targets under varying water turbidity levels illustrate that our method is realizable and robust.The precision is verified by comparing the results with the precise displacement data and calculating two error measures.The proposed method based on polarized optical flow can obtain accurate displacement information and a good recognition effect.Moving target segmentation based on the Otsu method further proves the superiority of the polarized optical flow under turbid water.This study is valuable for target detection and motion estimation in scattering environments.

Advanced 4-dimensional cone-beam computed tomography reconstruction by combining motion estimation, motioncompensated reconstruction, biomechanical modeling and deep learning

4-Dimensional cone-beam computed tomography(4D-CBCT)offers several key advantages over conventional 3DCBCT in moving target localization/delineation,structure de-blurring,target motion tracking,treatment dose accumulation and adaptive radiation therapy.However,the use of the 4D-CBCT in current radiation therapy practices has been limited,mostly due to its sub-optimal image quality from limited angular sampling of conebeam projections.In this study,we summarized the recent developments of 4D-CBCT reconstruction techniques for image quality improvement,and introduced our developments of a new 4D-CBCT reconstruction technique which features simultaneous motion estimation and image reconstruction(SMEIR).Based on the original SMEIR scheme,biomechanical modeling-guided SMEIR(SMEIR-Bio)was introduced to further improve the reconstruction accuracy of fine details in lung 4D-CBCTs.To improve the efficiency of reconstruction,we recently developed a U-net-based deformation-vector-field(DVF)optimization technique to leverage a population-based deep learning scheme to improve the accuracy of intra-lung DVFs(SMEIR-Unet),without explicit biomechanical modeling.Details of each of the SMEIR,SMEIR-Bio and SMEIR-Unet techniques were included in this study,along with the corresponding results comparing the reconstruction accuracy in terms of CBCT images and the DVFs.We also discussed the application prospects of the SMEIR-type techniques in image-guided radiation therapy and adaptive radiation therapy,and presented potential schemes on future developments to achieve faster and more accurate 4D-CBCT imaging.

MOVING OBJECT TRACKING IN DYNAMIC IMAGE SEQUENCE BASED ON ESTIMATION OF MOTION VECTORS OF FEATURE POINTS

An improved estimation of motion vectors of feature points is proposed for tracking moving objects of dynamic image sequence. Feature points are firstly extracted by the improved minimum intensity change （MIC） algorithm. The matching points of these feature points are then determined by adaptive rood pattern searching. Based on the random sample consensus （RANSAC） method, the background motion is finally compensated by the parameters of an affine transform of the background motion. With reasonable morphological filtering, the moving objects are completely extracted from the background, and then tracked accurately. Experimental results show that the improved method is successful on the motion background compensation and offers great promise in tracking moving objects of the dynamic image sequence.

Side Information Generation Based on Hierarchical Motion Estimation in Distributed Video Coding

The side information quality has an immense effect on the compression efficiency of the distributed video coding (DVC) sys- tem. This article, based on the hierarchical motion estimation (HME), proposes a new side information generation algorithm which is integrated into DVC system. First, forward motion estimation (FME) and bidirectional motion estimation (BME) on the basis of variable block size HME algorithm are used to acquire relatively accurate motion vectors. Second, a motion vector filter (MVF) is i...

Super-Resolution Reconstruction of Image Sequence Using Multiple Motion Estimation Fusion

Super-resolution reconstruction algorithm produces a high-resolution imagefrom a low-resolution image sequence. The accuracy and the stability of the motion estimation (ME)are essential for the whole restoration. In this paper, a new super-resolution reconstructionalgorithm is developed using a robust ME method, which fuses multiple estimated motion vectorswithin the sequence. The new algorithm has two major improvements compared with the previousresearch. First, instead of only two frames, the whole sequence is used to obtain a more accurateand stable estimation of the motion vector of each frame; second, the reliability of the ME isquantitatively measured and introduced into the cost function of the reconstruction algorithm. Thealgorithm is applied to both synthetic and real sequences, and the results are presented in thepaper.

New Robust Global Motion Estimation Approach Used in MPEG-4

This paper presents a new robust global motion estimation method based on pre-analysis of the video content. The novel idea in the proposed method, compared to classical robust statistics-based estimation methods, is to classify the video sequences into 3 classes based on the analysis of scene content before motion estimation. Different motion models and estimation methods are applied to different classes of image sequences. As a result, outliers can be identified and removed from the dominant motion estimate to solve the problem of inaccurate initial descending direction estimates associated with classical global motion estimation methods. The pre-analysis of scene content is based on the Spatial Temporal Gradient Scale (STGS) images derived from the original image sequences. The extra computation time for STGS-image-based pre-analysis of scene content is negligible compared to the overall speed and accuracy improvement achieved with the proposed method. Evaluations based on extensive experiments have shown that the proposed method significantly improves the speed of robust global motion estimation methods (saving about 50% of the execution time of classical methods).

MOTION-BASED REGION GROWING SEGMENTATION OF IMAGE SEQUENCES

This paper proposes a motion-based region growing segmentation scheme for the object-based video coding, which segments an image into homogeneous regions characterized by a coherent motion. It adopts a block matching algorithm to estimate motion vectors and uses morphological tools such as open-close by reconstruction and the region-growing version of the watershed algorithm for spatial segmentation to improve the temporal segmentation. In order to determine the reliable motion vectors, this paper also proposes a change detection algorithm and a multi-candidate pro- screening motion estimation method. Preliminary simulation results demonstrate that the proposed scheme is feasible. The main advantage of the scheme is its low computational load.

基于光流法的流体运动预测技术适用性研究

由于粒子图像测速技术在测量精度和时空分辨率上难以同时保证,而基于光流法的流体运动估计方法可以获得具有较高分辨率的稠密速度矢量场。为了评价该方法在流体运动估计上的适用性,采用流动情况由简单到复杂的开源粒子图像、标量图像、纹影图像序列对3种经典光流算法进行实验,在定性和定量层面评估了各算法的表现。结果表明:传统HS光流算法适用于大部分流动情况,然而在灰度梯度变化不明显时表现不佳;基于随机湍流传输模型的变分光流算法表现更为稳定,在HS目标函数中添加加权非局部平滑项后更适用于捕捉涡轮叶片纹影图像的尾迹区域和估计尾迹涡的输运速度。

基于姿态估计与深度图像的啦啦操运动员动作捕捉方法设计

为了对啦啦操运动员的动作进行有效的捕捉与识别,提出一种基于姿态估计与深度图像(PE-DI)的啦啦操运动员动作捕捉方法 .该方法采用深度摄像头收集啦啦操运动员的运动数据,并通过利用深度图像来增强三维空间的动作重建效果,接着利用姿态估计算法分析运动员的动作精度和稳定性.结果显示,动作识别率对比中,PE-DI算法、NN-BV算法、SPR-DL算法与Multi-Scale Feature算法在五次实验中的平均识别率分别为99.24%、97.18%、94.16%与90.21%.最后PE-DI算法运行下啦啦操运动员起势、跳跃、鞠躬与转身四个动作的均被捕捉划分于正确区域内,显著优于其他算法.实验结果表明,PE-DI算法能够对啦啦操运动员的动作进行较高精度的捕捉,为啦啦操运动员的训练与表演提供有效的技术支持.

基于运动偏移信息估计的图像去运动模糊算法

针对现有动态场景图像去运动模糊算法难以有效复原非均匀复合运动模糊的问题,通过引入相机曝光轨迹来表示模糊图像中包含的运动信息,并据此提出了一种新的运动偏移估计框架,用于对潜在清晰图像在多个离散时间点的像素运动偏移进行建模。结合估计出的运动偏移信息,提出了一种融合像素运动偏移信息的多尺度单幅图像去模糊网络框架。该框架通过可变形卷积在解码阶段对运动偏移信息进行融合,给定每一像素点不同的运动约束。经过网络的多尺度编解码结构,得到每一尺度上每一像素点的预测值,实现了端对端的模糊图像复原。在GoPro和HIDE数据集上的实验结果表明,该算法能有效改善图像质量,峰值信噪比平均提升1.9 dB,结构相似度平均提升0.03。

Electronic image stabilization system based on global feature tracking

A new robust electronic image stabilization system is presented, which involves feature-point, tracking based global motion estimation and Kalman filtering based motion compensation. First, global motion is estimated from the local motions of selected feature points. Considering the local moving objects or the inevitable mismatch, the matching validation, based on the stable relative distance between the points set is proposed, thus maintaining high accuracy and robustness. Next, the global motion parameters are accumulated for correction by Kalman filteration. The experimental result illustrates that the proposed system is effective to stabilize translational, rotational, and zooming jitter and robust to local motions.

主动电场线型电极阵列扫描成像方法研究

水下主动电场幅值法成像的灵感源于弱电鱼主动发射电信号识别周围环境这种生物机制。目前的主动水下成像技术在复杂黑暗的水下环境下仍有一定缺陷,因此一种新的解决方案被提出,即利用水下主动电场原理识别完成短距离目标物体复杂形状的成像。针对此前电极阵列覆盖有限、成像分辨率较小、对较大物体成像时电极数量多造成的成本较高以及缺少对复杂几何特征物体成像的实例等问题,设计了线型电极阵列连续运动扫描并采集数据的幅值成像平台,来对实验物体不同体积大小、不同材质及复杂形状进行成像,并对比结果,最后研究了如何根据横纵向幅值积分斜率曲线的最值点,提取成像图片中的物体位置和面积信息。实验结果证实了基于主动电场原理的线型电极阵列运动扫描幅值成像方法的有效性。与传统的主动电场成像方法相比,该方法不仅降低了硬件成本,还提高了成像的效率和分辨率。此外,该方法为主动电场成像技术的进一步工程化应用提供了一种实用的解决方案,优化了水下物体的成像过程,展示了其在复杂水域环境中的应用潜力。

A deep Koopman operator-based modelling approach for long-term prediction of dynamics with pixel-level measurements

Although previous studies have made some clear leap in learning latent dynamics from high-dimensional representations,the performances in terms of accuracy and inference time of long-term model prediction still need to be improved.In this study,a deep convolutional network based on the Koopman operator(CKNet)is proposed to model non-linear systems with pixel-level measurements for long-term prediction.CKNet adopts an autoencoder network architecture,consisting of an encoder to generate latent states and a linear dynamical model(i.e.,the Koopman operator)which evolves in the latent state space spanned by the encoder.The decoder is used to recover images from latent states.According to a multi-step ahead prediction loss function,the system matrices for approximating the Koopman operator are trained synchronously with the autoencoder in a mini-batch manner.In this manner,gradients can be synchronously transmitted to both the system matrices and the autoencoder to help the encoder self-adaptively tune the latent state space in the training process,and the resulting model is time-invariant in the latent space.Therefore,the proposed CKNet has the advantages of less inference time and high accuracy for long-term prediction.Experiments are per-formed on OpenAI Gym and Mujoco environments,including two and four non-linear forced dynamical systems with continuous action spaces.The experimental results show that CKNet has strong long-term prediction capabilities with sufficient precision.

基于语义与几何约束的动态SLAM系统

针对传统视觉SLAM在动态环境中出现的漂移、定位丢失和精度差等问题,提出一种基于语义先验信息和几何约束的面向动态环境的视觉SLAM系统。在传统ORB-SLAM2系统框架下加入语义分割模块,以提取物体的先验信息;利用先验的静态特征点进行几何约束,剔除图像中的动态特征点;针对剔除动态特征点会导致特征缺失等问题,加入图像补全线程,合成一个不包含动态物体的图像帧;利用合成帧求解相机位姿。实验结果表明,与传统算法相比,该算法在高动态环境中具有较高精度。

A Two-Stage Algorithm of High Resolution Image Alignment for Mobile Applications

Global motion estimation (GME) algorithms are widely applied to computer vision and video processing. In the previous works, the image resolutions are usually low for the real-time requirement (e.g. video stabilization). However, in some mobile devices applications (e.g. image sequence panoramic stitching), the high resolution is necessary to obtain satisfactory quality of panoramic image. However, the computational cost will become too expensive to be suitable for the low power consumption requirement of mobile device. The full search algorithm can obtain the global minimum with extremely computational cost, while the typical fast algorithms may suffer from the local minimum problem. This paper proposed a fast algorithm to deal with 2560 × 1920 high-resolution (HR) image sequences. The proposed method estimates the motion vector by a two-level coarse-to-fine scheme which only exploits sparse reference blocks (25 blocks in this paper) in each level to determine the global motion vector, thus the computational costs are significantly decreased. In order to increase the effective search range and robustness, the predictive motion vector (PMV) technique is used in this work. By the comparisons of computational complexity, the proposed algorithm costs less addition operations than the typical Three-Step Search algorithm (TSS) for estimating the global motion of the HR images without the local minimum problem. The quantitative evaluations show that our method is comparable to the full search algorithm (FSA) which is considered to be the golden baseline.

An Improved Optical Flow Method for Image Registration with Large-scale Movements

在这份报纸，为图象登记的一个改进光流动方法被建议。它以它与一个起始的运动评估者改进光流动方法的方法是新奇的：扩大了阶段关联技术(EPCT ) ，用补偿前者的缺乏的后者的优点。以一种更多的详细方式，光流动方法能到达亚象素精确性并且计算象啁啾并且倾斜一样的复杂失真模式，这能被说但是与大规模运动弱。因为 EPCT 用象素水平精确性盖住大翻译和旋转的大小并且在精明的负担是有效的，它能为光流动方法被当作一个好起始的运动评估者。测试证明了特别，这个改进方法将显著地与大规模运动为图象提高登记表演并且对随机的噪音柔韧。

Efficient Image Stitching in the Presence of Dynamic Objects and Structure Misalignment

This paper presents a new method for simultaneously eliminating visual artifacts caused by moving objects and structure misalignment in image stitching. Given that the input images are roughly aligned, our approach is implemented in two stages. In the first stage, we discover motions between input images, and then extract their corresponding regions through a multi-seed based region growing algorithm. In the second stage, with prior information provided by the extracted regions, we perform a graph cut optimization in gradient-domain to determine which pixels to use from each image to achieve seamless stitching. Our method is simple to implement and effective. The experimental results illustrate that the proposed approach can produce comparable or superior results in comparison with state-of-the-art methods.

Wavelet-Based Mixed-Resolution Coding Approach Incorporating with SPT for the Stereo Image

With the advances of display technology, three-dimensional(3-D) imaging systems are becoming increasingly popular. One way of stimulating 3-D perception is to use stereo pairs, a pair of images of the same scene acquired from different perspectives. Since there is an inherent redundancy between the images of a stereo pairs, data compression algorithms should be employed to represent stereo pairs efficiently. The proposed techniques generally use block-based disparity compensation. In order to get the higher compression ratio, this paper employs the wavelet-based mixed-resolution coding technique to incorporate with SPT-based disparity-compensation to compress the stereo image data. The mixed-resolution coding is a perceptually justified technique that is achieved by presenting one eye with a low-resolution image and the other with a high-resolution image. Psychophysical experiments show that the stereo image pairs with one high-resolution image and one low-resolution image provide almost the same stereo depth to that of a stereo image with two high-resolution images. By combining the mixed-resolution coding and SPT-based disparity-compensation techniques, one reference (left) high-resolution image can be compressed by a hierarchical wavelet transform followed by vector quantization and Huffman encoder. After two level wavelet decompositions, for the low-resolution right image and low-resolution left image, subspace projection technique using the fixed block size disparity compensation estimation is used. At the decoder, the low-resolution right subimage is estimated using the disparity from the low-resolution left subimage. A full-size reconstruction is obtained by upsampling a factor of 4 and reconstructing with the synthesis low pass filter. Finally, experimental results are presented, which show that our scheme achieves a PSNR gain (about 0.92dB) as compared to the current block-based disparity compensation coding techniques.

Real-Time Digital Image Stabilization Based on Contour

A new digital image stabilization method is proposed for real-time application based on image contour. The image intensities are projected to several gray levels by thresholding before extracting contour points. Matching position could be found using these contour points. All pixels are still used for refined matching near the matching position. This algorithm is more robust against changes in illumination and noise affection. The adaptive global motion judgement can remove the affection of intruding object. All those are realized on normally available PC.

An Electronic Image Stabilization Algorithm Based on Efficient Block Matching on the Bitplane

This paper proposes an electronic image stabilization algorithm based on efficient block matching on the plane. This algorithm uses a hexagonal search algorithm, and uses the bit-planes to estimate and compensate for the translational motion between video sequences at the same time;As for the rotary motion vector generated in the video sequences, in order to highlight the intensity change of the image sequence, the algorithm firstly conducts Laplace transform for the reference frame, then select a number of characteristics at the image edge to make block matching with the current frame, calculate and compensate for the rotational movement that may exist finally. Through theoretical analysis and simula-tion, we prove that, as for a mixed translational and rotational motion video sequences, the proposed algorithm can reduce required time for block matching computation ,while improving the accuracy of the electronic image stabilization.