Optimized Content-Aware Image Resizing with Merging and Improved Importance Diffusion

A simple and effective content-aware image resizing method is proposed based on the row / column merging and improved importance diffusion,which preserves the important regions in an image as well as the global visual effect. By repeatedly merging two rows / columns into one row / column or inserting a new row /column between two rows / columns, this method realizes image-resolution reduction and expansion. The importance of the merged row / column is promoted and diffused to four rows / columns around the merged one,which is to avoid the unwanted image distortions resulted from excessively merging of un-important regions. In addition,the proposed method introduces the direction of gradient vector in the low-pass filter to reduce the interference caused by complex texture background and protect important content better. Furthermore,according to human mechanics principles,different weights are given to the row and column direction components of gradient vectors which can obtain better global visual effect. Experimented results show that the proposed method satisfied in not only visual effect but also objective evaluation.

Real-time content-aware image resizing

Content-aware image resizing is a kind of new and effective approach for image resizing, which preserves image content well and does not cause obvious distortion when changing the aspect ratio of images. Recently, a seam based approach for content-aware image resizing was proposed by Avidan and Shamir. Their results are impressive, but because the method uses dynamic programming many times, it is slow. In this paper, we present a more efficient algorithm for seam based content-aware iraage resizing, which searches seams through establishing the matching relation between adjacent rows or columns. We give a linear algorithm to find the optimal matches within a weighted bipartite graph composed of the pixels in adjacent rows or columns. Therefore, our method is fast （e.g. our method needs only about 100 ms to reduce a 768x1024 Image＇s width to 1/3 while Avidan and Shamir＇s method needs 12 s）. This supports immediate image resizing whereas Avidan and Shamir＇s method requires a more costly pre-processing step to enable subsequent real-time processing. A fast method such as the one proposed will be also needed for future real-time video resizing applications.

An Efficient Detection Approach of Content Aware Image Resizing

Content aware image resizing(CAIR)is an excellent technology used widely for image retarget.It can also be used to tamper with images and bring the trust crisis of image content to the public.Once an image is processed by CAIR,the correlation of local neighborhood pixels will be destructive.Although local binary patterns(LBP)can effectively describe the local texture,it however cannot describe the magnitude information of local neighborhood pixels and is also vulnerable to noise.Therefore,to deal with the detection of CAIR,a novel forensic method based on improved local ternary patterns(ILTP)feature and gradient energy feature(GEF)is proposed in this paper.Firstly,the adaptive threshold of the original local ternary patterns(LTP)operator is improved,and the ILTP operator is used to describe the change of correlation among local neighborhood pixels caused by CAIR.Secondly,the histogram features of ILTP and the gradient energy features are extracted from the candidate image for CAIR forgery detection.Then,the ILTP features and the gradient energy features are concatenated into the combined features,and the combined features are used to train classifier.Finally support vector machine(SVM)is exploited as a classifier to be trained and tested by the above features in order to distinguish whether an image is subjected to CAIR or not.The candidate images are extracted from uncompressed color image database(UCID),then the training and testing sets are created.The experimental results with many test images show that the proposed method can detect CAIR tampering effectively,and that its performance is improved compared with other methods.It can achieve a better performance than the state-of-the-art approaches.

Cluster-Based Saliency-Guided Content-Aware Image Retargeting

Straightforward image resizing operators without considering image contents （e.g., uniform scaling） cannot usually produce satisfactory results, while content-aware image retargeting aims to arbitrarily change image size while preserving visually prominent features. In this paper, a cluster-based saliency-guided seam carving algorithm for content- aware image retargeting is proposed. To cope with the main drawback of the original seam carving algorithm relying on only gradient-based image importance map, we integrate a gradient-based map and a cluster-based saliency map to generate a more reliable importance map, resulting in better single image retargeting results. Experimental results have demonstrated the efficacy of the proposed algorithm.

基于LIR和GFNet的带钢表面缺陷识别

针对深度学习(deep learning,DL)模型处理带钢表面缺陷图像存在计算成本大、实时性差的问题,提出了一种基于可学习的图像调整器(learnable image resizer,LIR)和扫视-聚焦网络(glance and focus network,GFNet)的带钢表面缺陷分类方法。首先,针对DL模型在处理带钢表面缺陷图像时存在空间冗余的问题,提出GFNet驱动的带钢表面缺陷识别模型,其可以根据不同样本自适应分配计算资源,在模型推理阶段显著减少计算量;其次,提出LIR和GFNet联合训练的方法,调整图像大小的同时实现针对识别模型的特征增强;最后,收集整理了某钢铁企业冷轧薄板厂带钢表面缺陷数据集,利用所提方法进行分析。将残差网络(residual networks,ResNet)的ResNet-50模型作为主干网络,与原始ResNet-50比较,所提方法在不牺牲准确率的情况下,将单张图像的推断时间减少约3.58倍,计算量降低约6.11倍,从而验证了提出方法的有效性。

一种图像缩放的硬件加速电路

“深度学习算法”已经广泛应用于图像识别领域,深度学习需要做大量的数据传输和卷积运算,对速度和数据带宽的要求极高。本文的项目背景是用现场可编程门阵列(FPGA)方案实现基于多任务卷积神经网络(MTCNN)的人脸识别算法,图像缩放是MTCNN算法中的一个步骤,图像通常缓存在双倍速率同步动态随机存储器(DDR)中,对图像数据的处理受DDR带宽的限制,如果采用普通的算法势必导致图像缩放耗时较长,影响整个深度学习的效率。本文以该项目中DDR的16字节位宽数据传输为例,阐述了采用“位置查表法加特殊桶形移位器”的硬件算法实现数据输入输出的并行流水操作,经过测试本算法的速度是常规算法的6倍左右。

Fast Multi-Operator Image Resizing and Evaluation

Current multi-operator image resizing methods succeed in generating impressive results by using image similarity measure to guide the resizing process. An optimal operation path is found in the resizing space. However, their slow resizing speed caused by inefficient computation strategy of the bidirectional patch matching becomes a drawback in practical use. In this paper, we present a novel method to address this problem. By combining seam carving with scaling and cropping, our method can realize content-aware image resizing very fast. We define cost functions combing image energy and dominant color descriptor for all the operators to evaluate the damage to both local image content and global visual effect. Therefore our algorithm can automatically find an optimal sequence of operations to resize the image by using dynamic programming or greedy algorithm. We also extend our algorithm to indirect image resizing which can protect the aspect ratio of the dominant object in an image.

Review:A survey for image resizing

Image resizing is a key technique for displaying images on different devices, and has attracted much attention in the past few years. This paper reviews the image resizing methods proposed in recent years, gives a detailed comparison on their performance, and reveals the main challenges raised in several important issues such as preserving an important region, minimizing distortions, and improving efficiency. Furthermore, this paper discusses the research trends and points out the possible hotspots in this field. We believe this survey can give some guidance for researchers from relevant research areas, offering them an overall and novel view.

Image resizing by reconstruction from deep features

Traditional image resizing methods usually work in pixel space and use various saliency measures.The challenge is to adjust the image shape while trying to preserve important content.In this paper we perform image resizing in feature space using the deep layers of a neural network containing rich important semantic information.We directly adjust the image feature maps,extracted from a pre-trained classification network,and reconstruct the resized image using neuralnetwork based optimization.This novel approach leverages the hierarchical encoding of the network,and in particular,the high-level discriminative power of its deeper layers,that can recognize semantic regions and objects,thereby allowing maintenance of their aspect ratios.Our use of reconstruction from deep features results in less noticeable artifacts than use of imagespace resizing operators.We evaluate our method on benchmarks,compare it to alternative approaches,and demonstrate its strengths on challenging images.

基于重要度扩散和自适应采样的图像/视频缩放技术

提出了一种基于重要度扩散和自适应采样的图像和视频自适应缩放方法,它在整体概貌和重要区域保护之间进行折衷处理。重要度扩散函数将删除像素的重要性扩散至其邻域,以避免过多删除非重要区域而造成图像整体概貌失真。自适应采样函数则通过对各行和列像素的重要性进行权值的采样,以保护重要区域。此外,通过引入帧间一致性约束,该算法也适合于视频缩放。仿真实验结果表明:与剪切、Seam Carving等方法相比,本算法取得了较好的缩放效果。

基于扩展马尔科夫特征的Seam-Carving篡改检测

针对能够用于图像篡改的Seam-Carving技术,提出了一种基于扩展的马尔科夫特征的Seam-Carving篡改识别算法。该算法充分考虑了Seam-Carving操作导致的图像频域特征的变化,将传统的利用马尔科夫转移概率矩阵求取的图像特征和基于扩展的马尔科夫转移概率特征进行融合,而后利用支持向量机进行分类训练,从而达到有效识别基于Seam-Carving的图像篡改。实验结果表明,提出的方案性能优于传统的基于马尔科夫转移矩阵的特征选择方法以及现有的一些该类图像篡改检测方法。

基于随机置乱的内容感知图像缩放算法

针对非等比例缩放算法中线裁剪技术动态规划的复杂性,提出基于随机置乱的内容感知图像缩放算法。该算法利用视觉显著图随机重置能量序列的方式,使得在保护主体区域不被破坏的情况下,避免过度删除非重要区域所造成的图像局部失真。仿真结果表明,与线裁剪算法相比,本文算法在有效保护图像内容的同时,显著提高缩放速度,并能避免图像失真。

局部自适应非线性图像放大算法

已有的基于相关性检测的图像缩放算法所检测的边缘方向较少,并且只考虑了边缘特征的常数相关性,有时难以发掘像素之间所蕴含的关联性.为此,提出一种基于相关性检测的非线性图像放大算法.在检测图像中像素的相关性时,充分考虑像素之间潜在的相关方向;在检测相关性的过程中引入针对图像边缘特征的线性和二次相关性检测,用常数、线性或者二次函数来拟合像素之间的关系,从而更全面地描述像素之间的相关性,即图像的边缘特征,使得图像在放大过程中沿着图像的边缘特征进行,有效地消除了放大图像的模糊问题.实验结果表明,该算法具有良好的局部性,适合于GPU并行实现.

带锐度保持的斜投影B-样条图像缩放

基于B 样条的图像缩放相对于传统图像缩放方法 (最近邻点、双线性、双三次等 )的优势性在诸多文献和实验中得到了印证。但基于插值的B 样条和传统方法一样 ,没有考虑信息损失问题 ,是一种次优方法。为此Unser提出正交B 样条 ,并从理论上证明了其最优性。由于正交B 样条的采样核函数在阶数大于 3时难于获取 ,Lee发展了易于计算且保持了相当精度的斜投影B 样条。B 样条良好的连续性使图像的边缘在缩放过程中被模糊了。为了克服这一问题 ,提出了带锐度保持的斜投影B 样条图像缩放 ,在缩放过程中利用B 样条的递推性质对存在一阶或二阶微分过零点 (即有小边缘 )的采样区间缩小其积分范围至边缘附近 ,从而达到保持锐度的目的。实验结果表明该方法在保持缩放图像的连续性。

改进的图像缝雕刻算法

针对现有算法中对低频前景区域的处理效果欠佳的问题,提出一种改进的图像缝雕刻算法.它利用像素梯度信息对像素能量进行定义,然后提出最大能量最小函数来确定最佳的处理线路,再通过删除或添加单像素宽的线路,得到适合大小的目标图像.实验结果表明,该算法不仅能够保护图像中前景对象的显示效果,使之免受图像尺寸整体变化的影响,而且能够减少由于线路定位误差带来的前景对象扭曲.

基于主体区域保持的图像缩放算法

基于插值运算的缩放算法和经典的缝裁剪算法是两种常用的图像缩放算法,传统的缩放算法在缩放比例不一致的情况下其效果不佳,而缝裁剪算法在主体区域较大或者图像背景较为复杂时对图像的主体区域会造成一定破坏。针对以上问题,提出了一种基于主体区域保持的图像缩放算法,使用高斯差分对图像进行角点检测,利用角点产生凸包,根据凸包对图像进行主体区域检测,计算能量图并对位于主体区域像素点的能量给予相应的权重,根据权重的不同对主体区域进行不同程度的保护。实验结果表明,该算法能更好地保持图像主体区域。

主体大小能控的内容感知图像缩放

针对传统基于网格变形的图像缩放技术中可能出现的主体对象大小会随图像缩放而缩放的问题,提出一种既考虑网格形状,又考虑网格大小的形变量度量模型.在图像重要度约束下,通过简单的参数调整让用户选择性地控制主体对象的大小.重新定义图像重要度模型为梯度和显著度的加权平均,这样既考虑到人眼对图像的视觉关注,又考虑到对图像中结构信息的有效保护.在计算显著度时,改进基于稀疏特征的单分辨率模型为多分辨率模型,从而有效地保留了高分辨率下的边缘信息和低分辨率下的区域内部信息.实验结果表明,文中方法能更精确地识别出图像中的主体对象,并能在缩放时根据用户需求控制主体对象大小,同时很好地保持图像的重要特征.

基于罗伯茨梯度的彩色图像缩放新法

为了缩放图像而不改变图像中重要特征物的大小和比率,提出一种叫做缝络删增的缩放方法。一个缝络就是从顶到底或从左到右8-连接且仅连接一个像素的路径。最佳路径由图像的罗伯茨梯度函数来定义。通过在行列一个或二个方向重复删除或插入缝络就能改变图像的尺寸和比率。算法定义的缝络绕过了特征物边界,从而保护了图像的内容。理论分析和实验结果表明,该算法较其它算法能更有效地保护图像特征物。

显著物体形状结构保持的图像缩放方法

针对传统的缝裁剪图像缩放方法中可能出现对图像中显著物体形状结构的破坏问题,提出一种既考虑到显著物体内容保持又考虑到显著物体形状结构保持的新的图像缩放方法。该方法首先利用经典的图像显著度图模型,结合图像梯度直方图等信息构建形状结构更加清晰的图像重要度图;然后利用已构建的重要度图,对图像进行分块,按显著块的大小来确定缩放方法;最后结合经典缝裁剪方法和基于共形能量的变形方法对图像进行缩放。实验结果显示,该方法能够在图像缩放时更好地保持显著物体的内容和形状结构。

基于成对比较的内容感知图像缩放效果主观评估

为了对内容感知图像缩放方法的性能及效果进行评价,提出一种新的主观评估方法.该方法主要包括3个方面的工作:首先提出了一种结合非完整型成对比较法及随机化区组设计的用户测评实验方案,采用非完整型成对比较法,不要求每个被测主体对实验数据集上的所有比较对进行比较,减少了因被测主体注意力不稳定而对测试产生的影响,通过随机化区组设计,尽量减少不同被测主体进行测试的比较对间的差异;然后实时检测被测主体的测评结果中是否存在循环误判,若出现循环误判,则返回相应的比较对让用户再次确认选择结果,以此减少由被测主体注意力不集中而引起的误判;最后结合多种非参数统计方法来分析实验数据,先检测多个被测主体对不同方法结果的选择是否有相同的倾向,在得出一致性倾向显著的基础上,结合Friedman检验、Page检验及Hollander-Wolfe比较来给出不同方法的效果排序及分组.针对多种典型内容感知图像缩放方法进行广泛的用户测评实验,结果表明,实验数据有较高的一致性系数,得出的主观测评结果被较多的被测主体支持.