Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,w...Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.展开更多
Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real ...Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.展开更多
Dear Editor,3×3 Infrared imaging,generally,of low quality,plays an important role in security surveillance and target detection.In this letter,we improve the quality of infrared images by combining both hardware ...Dear Editor,3×3 Infrared imaging,generally,of low quality,plays an important role in security surveillance and target detection.In this letter,we improve the quality of infrared images by combining both hardware and software.To this end,an infrared light field imaging enhancement system is built for the first time,including a infrared light field imaging device,a large-scale infrared light field dataset(IRLF-WHU),and a progressive fusion network for infrared image enhancement(IR-PFNet).展开更多
Prostate cancers(PCa)have been reported to actively suppress antitumor immune responses by creating an immune-suppressive microenvironment.There is mounting evidence that PCas may undergo an‘‘Epithelial Immune Cell-...Prostate cancers(PCa)have been reported to actively suppress antitumor immune responses by creating an immune-suppressive microenvironment.There is mounting evidence that PCas may undergo an‘‘Epithelial Immune Cell-like Transition’’(EIT)by expressing molecules conventionally associated with immune cells(e.g.,a variety of cytokines/receptors,immune transcription factors,Ig motifs,and immune checkpoint molecules),which subsequently results in the suppression of anti-cancer immune activity within the tumor microenvironment.Recent progress within the field of immune therapy has underscored the importance of immune checkpoint molecules in cancer development,thus leading to the development of novel immunotherapeutic approaches.Here,we review the expression of select immune checkpoint molecules in PCa epithelial and associated immune cells,with particular emphasis on clinical data supporting the concept of an EIT-mediated phenotype in PCa.Furthermore,we summarize current advances in anti-immune checkpoint therapies,and provide perspectives on their potential applicability.展开更多
Most existing light field(LF)super-resolution(SR)methods either fail to fully use angular information or have an unbalanced performance distribution because they use parts of views.To address these issues,we propose a...Most existing light field(LF)super-resolution(SR)methods either fail to fully use angular information or have an unbalanced performance distribution because they use parts of views.To address these issues,we propose a novel integration network based on macro-pixel representation for the LF SR task,named MPIN.Restoring the entire LF image simultaneously,we couple the spatial and angular information by rearranging the four-dimensional LF image into a two-dimensional macro-pixel image.Then,two special convolutions are deployed to extract spatial and angular information,separately.To fully exploit spatial-angular correlations,the integration resblock is designed to merge the two kinds of information for mutual guidance,allowing our method to be angular-coherent.Under the macro-pixel representation,an angular shuffle layer is tailored to improve the spatial resolution of the macro-pixel image,which can effectively avoid aliasing.Extensive experiments on both synthetic and real-world LF datasets demonstrate that our method can achieve better performance than the state-of-the-art methods qualitatively and quantitatively.Moreover,the proposed method has an advantage in preserving the inherent epipolar structures of LF images with a balanced distribution of performance.展开更多
基金supported in part by the National Natural Science Foundation of China(62276192)。
文摘Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.
基金supported by the National Natural Science Foundation of China(61971165)the Key Research and Development Program of Hubei Province(2020BAB113)。
文摘Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.
文摘Dear Editor,3×3 Infrared imaging,generally,of low quality,plays an important role in security surveillance and target detection.In this letter,we improve the quality of infrared images by combining both hardware and software.To this end,an infrared light field imaging enhancement system is built for the first time,including a infrared light field imaging device,a large-scale infrared light field dataset(IRLF-WHU),and a progressive fusion network for infrared image enhancement(IR-PFNet).
文摘Prostate cancers(PCa)have been reported to actively suppress antitumor immune responses by creating an immune-suppressive microenvironment.There is mounting evidence that PCas may undergo an‘‘Epithelial Immune Cell-like Transition’’(EIT)by expressing molecules conventionally associated with immune cells(e.g.,a variety of cytokines/receptors,immune transcription factors,Ig motifs,and immune checkpoint molecules),which subsequently results in the suppression of anti-cancer immune activity within the tumor microenvironment.Recent progress within the field of immune therapy has underscored the importance of immune checkpoint molecules in cancer development,thus leading to the development of novel immunotherapeutic approaches.Here,we review the expression of select immune checkpoint molecules in PCa epithelial and associated immune cells,with particular emphasis on clinical data supporting the concept of an EIT-mediated phenotype in PCa.Furthermore,we summarize current advances in anti-immune checkpoint therapies,and provide perspectives on their potential applicability.
基金Project supported by the National Natural Science Foundation of China(No.61773295)。
文摘Most existing light field(LF)super-resolution(SR)methods either fail to fully use angular information or have an unbalanced performance distribution because they use parts of views.To address these issues,we propose a novel integration network based on macro-pixel representation for the LF SR task,named MPIN.Restoring the entire LF image simultaneously,we couple the spatial and angular information by rearranging the four-dimensional LF image into a two-dimensional macro-pixel image.Then,two special convolutions are deployed to extract spatial and angular information,separately.To fully exploit spatial-angular correlations,the integration resblock is designed to merge the two kinds of information for mutual guidance,allowing our method to be angular-coherent.Under the macro-pixel representation,an angular shuffle layer is tailored to improve the spatial resolution of the macro-pixel image,which can effectively avoid aliasing.Extensive experiments on both synthetic and real-world LF datasets demonstrate that our method can achieve better performance than the state-of-the-art methods qualitatively and quantitatively.Moreover,the proposed method has an advantage in preserving the inherent epipolar structures of LF images with a balanced distribution of performance.
