A survey on image and video stitching

Image/video stitching is a technology for solving the field of view(FOV)limitation of images/videos.It stitches multiple overlapping images/videos to generate a wide-FOV image/video,and has been used in various fields such as sports broadcasting,video surveillance,street view,and entertainment.This survey reviews image/video stitching algorithms,with a particular focus on those developed in recent years.Image stitching first calculates the corresponding relationships between multiple overlapping images,deforms and aligns the matched images,and then blends the aligned images to generate a wide-FOV image.A seamless method is always adopted to eliminate such potential flaws as ghosting and blurring caused by parallax or objects moving across the overlapping regions.Video stitching is the further extension of image stitching.It usually stitches selected frames of original videos to generate a stitching template by performing image stitching algorithms,and the subsequent frames can then be stitched according to the template.Video stitching is more complicated with moving objects or violent camera movement,because these factors introduce jitter,shakiness,ghosting,and blurring.Foreground detection technique is usually combined into stitching to eliminate ghosting and blurring,while video stabilization algorithms are adopted to solve the jitter and shakiness.This paper further discusses panoramic stitching as a special-extension of image/video stitching.Panoramic stitching is currently the most widely used application in stitching.This survey reviews the latest image/video stitching methods,and introduces the fundamental principles/advantages/weaknesses of image/video stitching algorithms.Image/video stitching faces long-term challenges such as wide baseline,large parallax,and low-texture problem in the overlapping region.New technologies may present new opportunities to address these issues,such as deep learning-based semantic correspondence,and 3D image stitching.Finally,this survey discusses the challenges of image/video stitching and proposes potential solutions.

Rapid Texture Mapping from Image Sequences for Building Geometry Model

An effective approach, mapping the texture for building model based on the digital photogrammetric theory, is proposed. The easily-acquired image sequences from digital video camera on helicopter are used as texture resource, and the correspondence between the space edge in building geometry model and its line feature in image sequences is determined semi-automatically. The experimental results in production of three-dimensional data for car navigation show us an attractive future both in efficiency and effect.

Dynamic Hyperlinker: Innovative Solution for 3D Video Content Search and Retrieval

Recently, 3D display technology, and content creation tools have been undergone rigorous development and as a result they have been widely adopted by home and professional users. 3D digital repositories are increasing and becoming available ubiquitously. However, searching and visualizing 3D content remains a great challenge. In this paper, we propose and present the development of a novel approach for creating hypervideos, which ease the 3D content search and retrieval. It is called the dynamic hyperlinker for 3D content search and retrieval process. It advances 3D multimedia navigability and searchability by creating dynamic links for selectable and clickable objects in the video scene whilst the user consumes the 3D video clip. The proposed system involves 3D video processing, such as detecting/tracking clickable objects, annotating objects, and metadata engineering including 3D content descriptive protocol. Such system attracts the attention from both home and professional users and more specifically broadcasters and digital content providers. The experiment is conducted on full parallax holoscopic 3D videos “also known as integral images”.

一种新的3－D小波变换图像编码方法

基于图像序列三维小波变换的视频编码技术是一种很有潜力的编码方法．本文针对目前实际中采用的三维小波变换只是一种准三维变换，不能提供对变换结果的统一描述等不足，推导了信号真正的三维小波变换的有关理论及其实现方法．在此基础上，我们将这种三维小波变换方法用于视频图像的编码，模拟结果表明这种编码结构具有良好性能，值得进一步的深入研究．

3D displays: toward holographic video displays of 3D images

As the fiat panel displays （Liquid Crystal Displays, AMOLED, etc.） reach near perfection in their viewing qualities and display areas, it is natural to seek the next level of displays, including 3D displays. There is a strong surge in 3D liquid crystal displays as a result of the successful movie Avatar. Most of these 3D displays involve the employment of special glasses that allow one view perspective for each of the eyes to achieve a depth perception. Such displays are not real 3D displays. In fact, these displays can only provide one viewing perspective for all viewers, regardless of the viewer＇s position. In addition, a fundamental viewing problem of focusing and accommodation exist that can lead to discomfort and fatigue for many viewers. In this paper, the authors review the current status of stereoscopic 3D displays and their problems. The authors will also discuss the possibility of using fiat panels for the display of both phase and intensity of video image information, leading to the ultimate display of 3D holographic video images. Many of the fundamental issues and limitations will be presented and discussed.

A retargeting method for stereoscopic 3D video

We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.

一种带图像修补的运动补偿视频插帧方法

为了减小运动补偿插帧时的"块效应",提出一种带图像修补的运动补偿插帧方法。该算法首先采用可变尺寸块的多级双向搜索,"由粗到细"逐级精确修正运动矢量;然后,对已搜索到合适运动矢量的块进行插值补偿,生成初始内插帧;对于仍没有合适运动估计结果的块并不生硬的内插,而是利用图像修补(image inpainting)技术进行填充,从而得到最终完整的内插帧。实验结果表明,该方法内插帧的主、客观质量较之对比算法均有所提高,且算法复杂度低,实用性强。

基于单目双焦及SIFT特征匹配的深度估计方法

提出一种基于单目双焦及SIFT特征匹配的深度估计方法。根据空间景物的深度在不同焦距下其成像的矢量位置和对应的焦距形成几何关系的原理,通过单相机获取两幅不同焦距下的图像后,运用SIFT算法对两幅图像进行特征提取和特征匹配,得出同一景物像素点距中心点的偏移位置比,从而通过几何关系公式计算出像素点的深度值,以此获取深度图。经实验验证了该方法的可行性,实验结果表明,使用该方法获取深度值仅需单台相机这一设备,方法简单易行,且成本低,具有广阔的应用范围。

三维动态偏移场模型及其在视频跟踪技术中的应用

基于二维动态偏移场模型,该文提出了动态偏移场模型的三维描述形式,实现了对目标沿景深方向运动状态的描述,并为失真图像序列的动态偏移提供了由平面偏移估计到空间运动状态估计的渐进描述。在视频跟踪系统中,较之其他视频跟踪技术,基于三维偏移场模型理论,该文实现了跟踪视频序列的稳定化和对被跟踪目标运动状态的估计。

快速3维坐标变换的绘制算法

目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355 d B,SSIM平均提高了0.001 15。结论改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。