Image decomposition and staircase effect reduction based on total generalized variation

Total variation （TV） is widely applied in image process-ing. The assumption of TV is that an image consists of piecewise constants, however, it suffers from the so-cal ed staircase effect. In order to reduce the staircase effect and preserve the edges when textures of image are extracted, a new image decomposition model is proposed in this paper. The proposed model is based on the to-tal generalized variation method which involves and balances the higher order of the structure. We also derive a numerical algorithm based on a primal-dual formulation that can be effectively imple-mented. Numerical experiments show that the proposed method can achieve a better trade-off between noise removal and texture extraction, while avoiding the staircase effect efficiently.

Image decomposition using adaptive regularization and div(BMO)

In order to avoid staircasing effect and preserve small scale texture information for the classical total variation regularization, a new minimization energy functional model for image decomposition is proposed. Firstly, an adaptive regularization based on the local feature of images is introduced to substitute total variational regularization. The oscillatory component containing texture and/or noise is modeled in generalized function space div （BMO）. And then, the existence and uniqueness of the minimizer for proposed model are proved. Finally, the gradient descent flow of the Euler-Lagrange equations for the new model is numerically implemented by using a finite difference method. Experiments show that the proposed model is very robust to noise, and the staircasing effect is avoided efficiently, while edges and textures are well remained.

Adaptive variational models for image decomposition combining staircase reduction and texture extraction

New models for image decomposition are proposed which separate an image into a cartoon, consisting only of geometric objects, and an oscillatory component, consisting of textures or noise. The proposed models are given in a variational formulation with adaptive regularization norms for both the cartoon and texture parts. The adaptive behavior preserves key features such as object boundaries and textures while avoiding staircasing in what should be smooth regions. This decomposition is computed by minimizing a convex functional which depends on the two variables u and v, alternatively in each variable. Experimental results and comparisons to validate the proposed models are presented.

Adaptive image decomposition method based on credible data fitting with local total variation

In this paper we present a novel image decomposition method via credible data fitting with local total variation filter. The oscillation rate is used to measure the image complexity and characteristics. The filter parameter can be determined by a fitting curve which is reconstructed by oscillation rate. In addition, the approximate Gaussian algorithm and integral image are used to reduce the algorithm computation and the sensitivity of the filter window selection. Experiments show the new method is better than the exist- ing methods.

Discriminative feature encoding for intrinsic image decomposition

Intrinsic image decomposition is an important and long-standing computer vision problem.Given an input image,recovering the physical scene properties is ill-posed.Several physically motivated priors have been used to restrict the solution space of the optimization problem for intrinsic image decomposition.This work takes advantage of deep learning,and shows that it can solve this challenging computer vision problem with high efficiency.The focus lies in the feature encoding phase to extract discriminative features for different intrinsic layers from an input image.To achieve this goal,we explore the distinctive characteristics of different intrinsic components in the high-dimensional feature embedding space.We define feature distribution divergence to efficiently separate the feature vectors of different intrinsic components.The feature distributions are also constrained to fit the real ones through a feature distribution consistency.In addition,a data refinement approach is provided to remove data inconsistency from the Sintel dataset,making it more suitable for intrinsic image decomposition.Our method is also extended to intrinsic video decomposition based on pixel-wise correspondences between adjacent frames.Experimental results indicate that our proposed network structure can outperform the existing state-of-the-art.

Image Smoothing Based on Image Decomposition and Sparse High Frequency Gradient

Image smoothing is a crucial image processing topic and has wide applications. For images with rich texture, most of the existing image smoothing methods are difficult to obtain significant texture removal performance because texture containing obvious edges and large gradient changes is easy to be preserved as the main edges. In this paper, we propose a novel framework （DSHFG） for image smoothing combined with the constraint of sparse high frequency gradient for texture images. First, we decompose the image into two components： a smooth component （constant component） and a non-smooth （high frequency） component. Second, we remove the non-smooth component containing high frequency gradient and smooth the other component combining with the constraint of sparse high frequency gradient. Experimental results demonstrate the proposed method is more competitive on efficiently texture removing than the state-of-the-art methods. What is more, our approach has a variety of applications including edge detection, detail magnification, image abstraction, and image composition.

Satellite Image Adaptive Restoration Using Periodic Plus Smooth Image Decomposition and Complex Wavelet Packet Transforms

A satellite image adaptive restoration method was developed that avoids ringing artifacts at the image boundary and retains oriented features. The method combines periodic plus smooth image decom- position with complex wavelet packet transforms. The framework first decomposes a degraded satellite im- age into the sum of a ＂periodic component＂ and a ＂smooth component＂. The Bayesian method is then used to estimate the modulation transfer function degradation parameters and the noise. The periodic component is deconvoluted using complex wavelet packet transforms with the deconvolution result of the periodic component then combined with the smooth component to get the final recovered result. Tests show that this strategy effectively avoids ringing artifacts while preserving local image details （especially directional tex- tures） without amplifying the noise. Quantitative comparisons illustrate that the results are comparable with previous methods. Another benefit is that this approach can process large satellite images with parallel processing, which is important for practical use.

A Nonlocal Total Variation Model for Image Decomposition: Illumination and Reflectance

In this paper,we study to use nonlocal bounded variation(NLBV)techniques to decompose an image intensity into the illumination and reflectance components.By considering spatial smoothness of the illumination component and nonlocal total variation(NLTV)of the reflectance component in the decomposition framework,an energy functional is constructed.We establish the theoretical results of the space of NLBV functions such as lower semicontinuity,approximation and compactness.These essential properties of NLBV functions are important tools to show the existence of solution of the proposed energy functional.Experimental results on both grey-level and color images are shown to illustrate the usefulness of the nonlocal total variation image decomposition model,and demonstrate the performance of the proposed method is better than the other testing methods.

A novel oil spill detection method from synthetic aperture radar imageries via a bidimensional empirical mode decomposition

Oil spills pose a major threat to ocean ecosystems and their health. Synthetic aperture radar（SAR） sensors can detect oil spills on the sea surface. These oil spills appear as dark spots in SAR images. However, dark formations can be caused by a number of phenomena. It is aimed to distinguishing oil spills or look-alike objects. A novel method based on a bidimensional empirical mode decomposition is proposed. The selected dark formations are first decomposed into several bidimensional intrinsic mode functions and the residue. Subsequently, 64 dimension feature sets are calculated using the Hilbert spectral analysis and five new features are extracted with a relief algorithm. Mahalanobis distances are then used for classification. Three data sets containing oil spills or look-alikes are used to test the accuracy rate of the method. The accuracy rate is more than 90%. The experimental results demonstrate that the novel method can detect oil spills validly and accurately.

DEANet: Decomposition Enhancement and Adjustment Network for Low-Light Image Enhancement

Poor illumination greatly affects the quality of obtained images.In this paper,a novel convolutional neural network named DEANet is proposed on the basis of Retinex for low-light image enhancement.DEANet combines the frequency and content information of images and is divided into three subnetworks:decomposition,enhancement,and adjustment networks,which perform image decomposition;denoising,contrast enhancement,and detail preservation;and image adjustment and generation,respectively.The model is trained on the public LOL dataset,and the experimental results show that it outperforms the existing state-of-the-art methods regarding visual effects and image quality.

基于图像分解的分层图像修复(英文)

We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform （DCT） in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.

RADepthNet:Reflectance-aware monocular depth estimation

Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy,which leads to inferior performance.Methods To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy,we propose RADepthNet,a novel reflectance-guided network that fuses boundary features.Specifically,our method predicts depth maps using the following three steps:(1)Intrinsic Image Decomposition.We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance.Through an ablation study,we demonstrate that the module can reduce the influence of illumination on depth estimation.(2)Boundary Detection.A boundary extraction module,consisting of an encoder,refinement block,and upsample block,was proposed to better predict the depth at object boundaries utilizing gradient constraints.(3)Depth Prediction Module.We use an encoder different from(2)to obtain depth features from the reflectance map and fuse boundary features to predict depth.In addition,we proposed FIFADataset,a depth-estimation dataset applied in soccer scenarios.Results Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.

Interactive lighting editing system for single indoor low-light scene images with corresponding depth maps

We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light indoor scene.Our method not only achieves plausible global relighting but also enhances the local details of the complicated scene according to the spatially-varying spherical harmonic lighting,which only requires a single RGB image along with a corresponding depth map.To this end,we first present a joint optimization algorithm,which is based on the geometric optimization of the depth map and intrinsic image decomposition avoiding texture-copy,for refining the depth map and obtaining the shading map.Then we propose a lighting estimation method based on spherical harmonic lighting,which not only achieves the global illumination estimation of the scene,but also further enhances local details of the complicated scene.Finally,we use a simple and intuitive interactive method to edit the environment lighting map to adjust lighting and relight the scene.Through extensive experimental results,we demonstrate that our proposed approach is simple and intuitive for relighting the low-light indoor scene,and achieve state-of-the-art results.

A non-Lambertian photometric stereo under perspective projection

Under the perspective projection assumption,non-Lambertian photometric stereo is a highly non-linear problem.In this study,we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection.By decomposing the images into diffuse and specular components,we compute the surface normal and reflectance simultaneously.We also propose a variational formulation that is robust and useful for surface reconstruction.The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.