A new matting algorithm based on color distance and differential distance is proposed to deal with the problem that many matting methods perform poorly with complex natural images.The proposed method combines local sa...A new matting algorithm based on color distance and differential distance is proposed to deal with the problem that many matting methods perform poorly with complex natural images.The proposed method combines local sampling with global sampling to select foreground and background pairs for unknown pixels and then a new cost function is constructed based on color distance and differential distance to further optimize the selected sample pairs.Finally,a quadratic objective function is used based on matte Laplacian coming from KNN matting which is added with texture feature.Through experiments on various test images,it is confirmed that the results obtained by the proposed method are more accurate than those obtained by traditional methods.The four-error-metrics comparison on benchmark dataset among several algorithms also proves the effectiveness of the proposed method.展开更多
This paper proposes a Maxkov Random Field （MRF） model-based approach to natural image matting with complex scenes. After the trimap for matting is given manually, the unknown region is roughly segmented into several...This paper proposes a Maxkov Random Field （MRF） model-based approach to natural image matting with complex scenes. After the trimap for matting is given manually, the unknown region is roughly segmented into several joint sub-regions. In each sub-region, we partition the colors of neighboring background or foreground pixels into several clusters in RGB color space and assign matting label to each unknown pixel. All the labels are modelled as an MRF and the matting problem is then formulated as a maximum a posteriori （MAP） estimation problem. Simulated annealing is used to find the optimal MAP estimation. The better results can be obtained under the same user-interactions when images are complex. Results of natural image matting experiments performed on complex images using this approach are shown and compared in this paper.展开更多
Pre-processing is an important step in digital image matting, which aims to classify more accurate foreground and background pixels from the unknown region of the input three-region mask （Trimap）. This step has no r...Pre-processing is an important step in digital image matting, which aims to classify more accurate foreground and background pixels from the unknown region of the input three-region mask （Trimap）. This step has no relation with the well-known matting equation and only compares color differences between the current unknown pixel and those known pixels. These newly classified pure pixels are then fed to the matting process as samples to improve the quality of the final matte. However, in the research field of image matting, the importance of pre-processing step is still blurry. Moreover, there are no corresponding review articles for this step, and the quantitative comparison of Trimap and alpha mattes after this step still remains unsolved. In this paper, the necessity and the importance of pre-processing step in image matting are firstly discussed in details. Next, current pre-processing methods are introduced by using the following two categories： static thresholding methods and dynamic thresholding methods. Analyses and experimental results show that static thresholding methods, especially the most popular iterative method, can make accurate pixel classifications in those general Trimaps with relatively fewer unknown pixels. However, in a much larger Trimap, there methods are limited by the conservative color and spatial thresholds. In contrast, dynamic thresholding methods can make much aggressive classifications on much difficult cases, but still strongly suffer from noises and false classifications. In addition, the sharp boundary detector is further discussed as a prior of pure pixels. Finally, summaries and a more effective approach are presented for pre-processing compared with the existing methods.展开更多
We propose a novel end-to-end deep learning framework, the Joint Matting Network(JMNet), to automatically generate alpha mattes for human images.We utilize the intrinsic structures of the human body as seen in images ...We propose a novel end-to-end deep learning framework, the Joint Matting Network(JMNet), to automatically generate alpha mattes for human images.We utilize the intrinsic structures of the human body as seen in images by introducing a pose estimation module,which can provide both global structural guidance and a local attention focus for the matting task. Our network model includes a pose network, a trimap network, a matting network, and a shared encoder to extract features for the above three networks. We also append a trimap refinement module and utilize gradient loss to provide a sharper alpha matte. Extensive experiments have shown that our method outperforms state-of-theart human matting techniques;the shared encoder leads to better performance and lower memory costs.Our model can process real images downloaded from the Internet for use in composition applications.展开更多
Image matting is an essential image processing technology due to its wide range of applications.Sampling-based image matting is one of the main branches of image matting research that estimates alpha mattes by selecti...Image matting is an essential image processing technology due to its wide range of applications.Sampling-based image matting is one of the main branches of image matting research that estimates alpha mattes by selecting the best pixel pairs.It is essentially a large-scale multi-peak optimization problem of pixel pairs.Previous study shows that particle swarm optimization(PSO)can effectively optimize the pixel pairs.However,it still suffers from premature convergence problem which often occurs in pixel pair optimization that involves a large number of local optima.To address this problem,this work presents a parameter-free strategy for PSO called adaptive convergence speed controller(ACSC).ACSC monitors and conditionally controls the particles by competitive pixel pair recombination operator(CPPRO)and pixel pair reset operator(PPRO)during the iteration.ACSC performs CPPRO to improve the competitiveness of a particle when the performance of most of the pixel pairs is worse than that of the best-so-far solution.PPRO is performed to avoid premature convergence when the alpha mattes regarding two selecleu particles ae liglly siimilau.Expeiilfental results show that ACSC significantly enhances the performance of PSO for image matting and provides competitive alpha mattes comparing with state-of-the-art evolutionary algorithms.展开更多
Detection and segmentation of defocus blur is a challenging task in digital imaging applications as the blurry images comprise of blur and sharp regions that wrap significant information and require effective methods ...Detection and segmentation of defocus blur is a challenging task in digital imaging applications as the blurry images comprise of blur and sharp regions that wrap significant information and require effective methods for information extraction.Existing defocus blur detection and segmentation methods have several limitations i.e.,discriminating sharp smooth and blurred smooth regions,low recognition rate in noisy images,and high computational cost without having any prior knowledge of images i.e.,blur degree and camera configuration.Hence,there exists a dire need to develop an effective method for defocus blur detection,and segmentation robust to the above-mentioned limitations.This paper presents a novel features descriptor local directional mean patterns(LDMP)for defocus blur detection and employ KNN matting over the detected LDMP-Trimap for the robust segmentation of sharp and blur regions.We argue/hypothesize that most of the image fields located in blurry regions have significantly less specific local patterns than those in the sharp regions,therefore,proposed LDMP features descriptor should reliably detect the defocus blurred regions.The fusion of LDMP features with KNN matting provides superior performance in terms of obtaining high-quality segmented regions in the image.Additionally,the proposed LDMP features descriptor is robust to noise and successfully detects defocus blur in high-dense noisy images.Experimental results on Shi and Zhao datasets demonstrate the effectiveness of the proposed method in terms of defocus blur detection.Evaluation and comparative analysis signify that our method achieves superior segmentation performance and low computational cost of 15 seconds.展开更多
Image matting is to estimate the opacity of foreground objects from an image. A few deep learning based methods have been proposed for image matting and perform well in capturing spatially close information. However, ...Image matting is to estimate the opacity of foreground objects from an image. A few deep learning based methods have been proposed for image matting and perform well in capturing spatially close information. However, these methods fail to capture global contextual information, which has been proved essential in improving matting performance. This is because a matting image may be up to several megapixels, which is too big for a learning-based network to capture global contextual information due to the limit size of a receptive field. Although uniformly downsampling the matting image can alleviate this problem, it may result in the degradation of matting performance. To solve this problem, we introduce a natural image matting with the attended global context method to extract global contextual information from the whole image, and to condense them into a suitable size for learning-based network. Specifically, we first leverage a deformable sampling layer to obtain condensed foreground and background attended images respectively. Then, we utilize a contextual attention layer to extract information related to unknown regions from condensed foreground and background images generated by a deformable sampling layer. Besides, our network predicts a background as well as the alpha matte to obtain more purified foreground, which contributes to better qualitative performance in composition. Comprehensive experiments show that our method achieves competitive performance on both Composition-1k and the alphamatting.com benchmark quantitatively and qualitatively.展开更多
While the geodetic excitationχ(t)of polar motion p(t)is essential to improve our understanding of global mass redistributions and relative motions with respect to the terrestrial frame,the widely adopted method to de...While the geodetic excitationχ(t)of polar motion p(t)is essential to improve our understanding of global mass redistributions and relative motions with respect to the terrestrial frame,the widely adopted method to deriveχ(t)from p(t)has biases in both amplitude and phase responses.This study has developed a new simple but more accurate method based on the combination of the frequency-and time-domain Liouville's equation(FTLE).The FTLE method has been validated not only with 6-h sampled synthetic excitation series but also with daily and 6-h sampled polar motion measurements as well asχ(t)produced by the interactive webpage tool of the International Earth Rotation and Reference Systems Service(IERS).Numerical comparisons demonstrate thatχ(t)derived from the FTLE method has superior performances in both the time and frequency domains with respect to that obtained from the widely adopted method or the IERS webpage tool,provided that the input p(t)series has a length around or more than 25 years,which presents no practical limitations since the necessary polar motion data are readily available.The FTLE code is provided in the form of Mat Lab function.展开更多
Interference management is one of the most important issues in the device-to-device(D2D)-enabled heterogeneous cellular networks(HetCNets)due to the coexistence of massive cellular and D2D devices in which D2D devices...Interference management is one of the most important issues in the device-to-device(D2D)-enabled heterogeneous cellular networks(HetCNets)due to the coexistence of massive cellular and D2D devices in which D2D devices reuse the cellular spectrum.To alleviate the interference,an efficient interference management way is to set exclusion zones around the cellular receivers.In this paper,we adopt a stochastic geometry approach to analyze the outage probabilities of cellular and D2D users in the D2D-enabled HetCNets.The main difficulties contain three aspects:1)how to model the location randomness of base stations,cellular and D2D users in practical networks;2)how to capture the randomness and interrelation of cellular and D2D transmissions due to the existence of random exclusion zones;3)how to characterize the different types of interference and their impacts on the outage probabilities of cellular and D2D users.We then run extensive Monte-Carlo simulations which manifest that our theoretical model is very accurate.展开更多
基金Supported by the National Natural Science Foundation of China(No.61133009,U1304616)
文摘A new matting algorithm based on color distance and differential distance is proposed to deal with the problem that many matting methods perform poorly with complex natural images.The proposed method combines local sampling with global sampling to select foreground and background pairs for unknown pixels and then a new cost function is constructed based on color distance and differential distance to further optimize the selected sample pairs.Finally,a quadratic objective function is used based on matte Laplacian coming from KNN matting which is added with texture feature.Through experiments on various test images,it is confirmed that the results obtained by the proposed method are more accurate than those obtained by traditional methods.The four-error-metrics comparison on benchmark dataset among several algorithms also proves the effectiveness of the proposed method.
基金This work was supported by the National Natural Science Foundation of China under Grant No. 600330107 Zhejiang Provincial Natural Science Foundation of China under Grant No, Y105324 and Planned Program of Science and Technology Department of Zhejiang Province, China （Grant No. 2006C31065）,
文摘This paper proposes a Maxkov Random Field （MRF） model-based approach to natural image matting with complex scenes. After the trimap for matting is given manually, the unknown region is roughly segmented into several joint sub-regions. In each sub-region, we partition the colors of neighboring background or foreground pixels into several clusters in RGB color space and assign matting label to each unknown pixel. All the labels are modelled as an MRF and the matting problem is then formulated as a maximum a posteriori （MAP） estimation problem. Simulated annealing is used to find the optimal MAP estimation. The better results can be obtained under the same user-interactions when images are complex. Results of natural image matting experiments performed on complex images using this approach are shown and compared in this paper.
文摘Pre-processing is an important step in digital image matting, which aims to classify more accurate foreground and background pixels from the unknown region of the input three-region mask （Trimap）. This step has no relation with the well-known matting equation and only compares color differences between the current unknown pixel and those known pixels. These newly classified pure pixels are then fed to the matting process as samples to improve the quality of the final matte. However, in the research field of image matting, the importance of pre-processing step is still blurry. Moreover, there are no corresponding review articles for this step, and the quantitative comparison of Trimap and alpha mattes after this step still remains unsolved. In this paper, the necessity and the importance of pre-processing step in image matting are firstly discussed in details. Next, current pre-processing methods are introduced by using the following two categories： static thresholding methods and dynamic thresholding methods. Analyses and experimental results show that static thresholding methods, especially the most popular iterative method, can make accurate pixel classifications in those general Trimaps with relatively fewer unknown pixels. However, in a much larger Trimap, there methods are limited by the conservative color and spatial thresholds. In contrast, dynamic thresholding methods can make much aggressive classifications on much difficult cases, but still strongly suffer from noises and false classifications. In addition, the sharp boundary detector is further discussed as a prior of pure pixels. Finally, summaries and a more effective approach are presented for pre-processing compared with the existing methods.
基金supported by National Natural Science Foundation of China(Grant Nos.61561146393 and61521002)supported by a Victoria Early-Career Research Excellence Award。
文摘We propose a novel end-to-end deep learning framework, the Joint Matting Network(JMNet), to automatically generate alpha mattes for human images.We utilize the intrinsic structures of the human body as seen in images by introducing a pose estimation module,which can provide both global structural guidance and a local attention focus for the matting task. Our network model includes a pose network, a trimap network, a matting network, and a shared encoder to extract features for the above three networks. We also append a trimap refinement module and utilize gradient loss to provide a sharper alpha matte. Extensive experiments have shown that our method outperforms state-of-theart human matting techniques;the shared encoder leads to better performance and lower memory costs.Our model can process real images downloaded from the Internet for use in composition applications.
基金supported by the National Nat-ural Science Foundation of China(Grant Nos.61772225,61876207,61502088)National Key R&D Program of China(2018YFCO823803,2018YFCO823802)+7 种基金Zhongshan Science and Technology Research Project of Social welfare(2019B2010)Guangdong Natural Science Fundsfor Distinguished Young Scholar(2014A030306050)Guangdong High-level personnel of special support program(2014TQ01X664)International Cooperator Project of Guangzhou(201807010047)National Natural Scicnce Foundation of Guangdong(2018B030311046)Guangdong University Key Platforms and Research Projects(2018KZDXMO66,2017KZDXM081,2015KQNCX153)Guangzhou Science and Technology Projects(201802010007,201804010276)Youth science and technologytalents cultivating object of Guizhou province(Qian education cooperation KY word[2016]165).
文摘Image matting is an essential image processing technology due to its wide range of applications.Sampling-based image matting is one of the main branches of image matting research that estimates alpha mattes by selecting the best pixel pairs.It is essentially a large-scale multi-peak optimization problem of pixel pairs.Previous study shows that particle swarm optimization(PSO)can effectively optimize the pixel pairs.However,it still suffers from premature convergence problem which often occurs in pixel pair optimization that involves a large number of local optima.To address this problem,this work presents a parameter-free strategy for PSO called adaptive convergence speed controller(ACSC).ACSC monitors and conditionally controls the particles by competitive pixel pair recombination operator(CPPRO)and pixel pair reset operator(PPRO)during the iteration.ACSC performs CPPRO to improve the competitiveness of a particle when the performance of most of the pixel pairs is worse than that of the best-so-far solution.PPRO is performed to avoid premature convergence when the alpha mattes regarding two selecleu particles ae liglly siimilau.Expeiilfental results show that ACSC significantly enhances the performance of PSO for image matting and provides competitive alpha mattes comparing with state-of-the-art evolutionary algorithms.
基金This work was supported and funded by the Directorate ASR&TD of UET-Taxila.
文摘Detection and segmentation of defocus blur is a challenging task in digital imaging applications as the blurry images comprise of blur and sharp regions that wrap significant information and require effective methods for information extraction.Existing defocus blur detection and segmentation methods have several limitations i.e.,discriminating sharp smooth and blurred smooth regions,low recognition rate in noisy images,and high computational cost without having any prior knowledge of images i.e.,blur degree and camera configuration.Hence,there exists a dire need to develop an effective method for defocus blur detection,and segmentation robust to the above-mentioned limitations.This paper presents a novel features descriptor local directional mean patterns(LDMP)for defocus blur detection and employ KNN matting over the detected LDMP-Trimap for the robust segmentation of sharp and blur regions.We argue/hypothesize that most of the image fields located in blurry regions have significantly less specific local patterns than those in the sharp regions,therefore,proposed LDMP features descriptor should reliably detect the defocus blurred regions.The fusion of LDMP features with KNN matting provides superior performance in terms of obtaining high-quality segmented regions in the image.Additionally,the proposed LDMP features descriptor is robust to noise and successfully detects defocus blur in high-dense noisy images.Experimental results on Shi and Zhao datasets demonstrate the effectiveness of the proposed method in terms of defocus blur detection.Evaluation and comparative analysis signify that our method achieves superior segmentation performance and low computational cost of 15 seconds.
基金supported by the National Natural Science Foundation of China under Grant No.62076162the Shanghai Municipal Science and Technology Major Project under Grant Nos.2021SHZDZX0102 and 20511100300.
文摘Image matting is to estimate the opacity of foreground objects from an image. A few deep learning based methods have been proposed for image matting and perform well in capturing spatially close information. However, these methods fail to capture global contextual information, which has been proved essential in improving matting performance. This is because a matting image may be up to several megapixels, which is too big for a learning-based network to capture global contextual information due to the limit size of a receptive field. Although uniformly downsampling the matting image can alleviate this problem, it may result in the degradation of matting performance. To solve this problem, we introduce a natural image matting with the attended global context method to extract global contextual information from the whole image, and to condense them into a suitable size for learning-based network. Specifically, we first leverage a deformable sampling layer to obtain condensed foreground and background attended images respectively. Then, we utilize a contextual attention layer to extract information related to unknown regions from condensed foreground and background images generated by a deformable sampling layer. Besides, our network predicts a background as well as the alpha matte to obtain more purified foreground, which contributes to better qualitative performance in composition. Comprehensive experiments show that our method achieves competitive performance on both Composition-1k and the alphamatting.com benchmark quantitatively and qualitatively.
基金supported by the National Natural Science Foundation of China(grant numbers 41874025 and 41474022)。
文摘While the geodetic excitationχ(t)of polar motion p(t)is essential to improve our understanding of global mass redistributions and relative motions with respect to the terrestrial frame,the widely adopted method to deriveχ(t)from p(t)has biases in both amplitude and phase responses.This study has developed a new simple but more accurate method based on the combination of the frequency-and time-domain Liouville's equation(FTLE).The FTLE method has been validated not only with 6-h sampled synthetic excitation series but also with daily and 6-h sampled polar motion measurements as well asχ(t)produced by the interactive webpage tool of the International Earth Rotation and Reference Systems Service(IERS).Numerical comparisons demonstrate thatχ(t)derived from the FTLE method has superior performances in both the time and frequency domains with respect to that obtained from the widely adopted method or the IERS webpage tool,provided that the input p(t)series has a length around or more than 25 years,which presents no practical limitations since the necessary polar motion data are readily available.The FTLE code is provided in the form of Mat Lab function.
基金This work is funded in part by the Science and Technology Development Fund,Macao SAR(Grant Nos.0093/2022/A2,0076/2022/A2 and 0008/2022/AGJ)in part by the National Nature Science Foundation of China(Grant No.61872452)+3 种基金in part by Special fund for Dongguan’s Rural Revitalization Strategy in 2021(Grant No.20211800400102)in part by Dongguan Special Commissioner Project(Grant No.20211800500182)in part by Guangdong-Dongguan Joint Fund for Basic and Applied Research of Guangdong Province(Grant No.2020A1515110162)in part by University Special Fund of Guangdong Provincial Department of Education(Grant No.2022ZDZX1073).
文摘Interference management is one of the most important issues in the device-to-device(D2D)-enabled heterogeneous cellular networks(HetCNets)due to the coexistence of massive cellular and D2D devices in which D2D devices reuse the cellular spectrum.To alleviate the interference,an efficient interference management way is to set exclusion zones around the cellular receivers.In this paper,we adopt a stochastic geometry approach to analyze the outage probabilities of cellular and D2D users in the D2D-enabled HetCNets.The main difficulties contain three aspects:1)how to model the location randomness of base stations,cellular and D2D users in practical networks;2)how to capture the randomness and interrelation of cellular and D2D transmissions due to the existence of random exclusion zones;3)how to characterize the different types of interference and their impacts on the outage probabilities of cellular and D2D users.We then run extensive Monte-Carlo simulations which manifest that our theoretical model is very accurate.
