FISS GAN:A Generative Adversarial Network for Foggy Image Semantic Segmentation

Because pixel values of foggy images are irregularly higher than those of images captured in normal weather(clear images),it is difficult to extract and express their texture.No method has previously been developed to directly explore the relationship between foggy images and semantic segmentation images.We investigated this relationship and propose a generative adversarial network(GAN)for foggy image semantic segmentation(FISS GAN),which contains two parts:an edge GAN and a semantic segmentation GAN.The edge GAN is designed to generate edge information from foggy images to provide auxiliary information to the semantic segmentation GAN.The semantic segmentation GAN is designed to extract and express the texture of foggy images and generate semantic segmentation images.Experiments on foggy cityscapes datasets and foggy driving datasets indicated that FISS GAN achieved state-of-the-art performance.

Semantic image annotation based on GMM and random walk model

Automatic image annotation has been an active topic of research in computer vision and pattern recognition for decades.A two stage automatic image annotation method based on Gaussian mixture model(GMM) and random walk model(abbreviated as GMM-RW) is presented.To start with,GMM fitted by the rival penalized expectation maximization(RPEM) algorithm is employed to estimate the posterior probabilities of each annotation keyword.Subsequently,a random walk process over the constructed label similarity graph is implemented to further mine the potential correlations of the candidate annotations so as to capture the refining results,which plays a crucial role in semantic based image retrieval.The contributions exhibited in this work are multifold.First,GMM is exploited to capture the initial semantic annotations,especially the RPEM algorithm is utilized to train the model that can determine the number of components in GMM automatically.Second,a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels,which is able to avoid the phenomena of polysemy and synonym efficiently during the image annotation process.Third,the random walk is implemented over the constructed label graph to further refine the candidate set of annotations generated by GMM.Conducted experiments on the standard Corel5 k demonstrate that GMM-RW is significantly more effective than several state-of-the-arts regarding their effectiveness and efficiency in the task of automatic image annotation.

Image Semantic Segmentation Approach for Studying Human Behavior on Image Data

Image semantic segmentation is an essential technique for studying human behavior through image data.This paper proposes an image semantic segmentation method for human behavior research.Firstly,an end-to-end convolutional neural network architecture is proposed,which consists of a depth-separable jump-connected fully convolutional network and a conditional random field network;then jump-connected convolution is used to classify each pixel in the image,and an image semantic segmentation method based on convolu-tional neural network is proposed;and then a conditional random field network is used to improve the effect of image segmentation of hu-man behavior and a linear modeling and nonlinear modeling method based on the semantic segmentation of conditional random field im-age is proposed.Finally,using the proposed image segmentation network,the input entrepreneurial image data is semantically segmented to obtain the contour features of the person;and the segmentation of the images in the medical field.The experimental results show that the image semantic segmentation method is effective.It is a new way to use image data to study human behavior and can be extended to other research areas.

Low-light image enhancement algorithm using a residual network with semantic information

Aiming to solve the poor performance of low illumination enhancement algorithms on uneven illumination images,a low-light image enhancement(LIME)algorithm based on a residual network was proposed.The algorithm constructs a deep network that uses residual modules to extract image feature information and semantic modules to extract image semantic information from different levels.Moreover,a composite loss function was also designed for the process of low illumination image enhancement,which dynamically evaluated the loss of an enhanced image from three factors of color,structure,and gradient.It ensures that the model can correctly enhance the image features according to the image semantics,so that the enhancement results are more in line with the human visual experience.Experimental results show that compared with the state-of-the-art algorithms,the semantic-driven residual low-light network(SRLLN)can effectively improve the quality of low illumination images,and achieve better subjective and objective evaluation indexes on different types of images.

Understanding satellite images:a data mining module for Sentinel images

The increased number of free and open Sentinel satellite images has led to new applications of these data.Among them is the systematic classification of land cover/use types based on patterns of settlements or agriculture recorded by these images,in particular,the identification and quantification of their temporal changes.In this paper,we will present guidelines and practical examples of how to obtain rapid and reliable image patch labelling results and their validation based on data mining techniques for detecting these temporal changes,and presenting these as classification maps and/or statistical analytics.This represents a new systematic validation approach for semantic image content verification.We will focus on a number of different scenarios proposed by the user community using Sentinel data.From a large number of potential use cases,we selected three main cases,namely forest monitoring,flood monitoring,and macro-economics/urban monitoring.

Learning deep representations for semantic image parsing： a comprehensive overview

Semantic image parsing, which refers to the pro- cess of decomposing images into semantic regions and constructing the structure representation of the input, has re- cently aroused widespread interest in the field of computer vision. The recent application of deep representation learning has driven this field into a new stage of development. In this paper, we summarize three aspects of the progress of research on semantic image parsing, i.e., category-level semantic segmentation, instance-level semantic segmentation, and beyond segmentation. Specifically, we first review the general frameworks for each task and introduce the relevant variants. The advantages and limitations of each method are also discussed. Moreover, we present a comprehensive comparison of different benchmark datasets and evaluation metrics. Finally, we explore the future trends and challenges of semantic image parsing.

Image Tagging by Semantic Neighbor Learning Using User-Contributed Social Image Datasets

The explosive increase in the number of images on the Internet has brought with it the great challenge of how to effectively index, retrieve, and organize these resources. Assigning proper tags to the visual content is key to the success of many applications such as image retrieval and content mining. Although recent years have witnessed many advances in image tagging, these methods have limitations when applied to high-quality and large-scale training data that are expensive to obtain. In this paper, we propose a novel semantic neighbor learning method based on user-contributed social image datasets that can be acquired from the Web＇s inexhaustible social image content. In contrast to existing image tagging approaches that rely on high-quality image-tag supervision, we acquire weak supervision of our neighbor learning method by progressive neighborhood retrieval from noisy and diverse user-contributed image collections. The retrieved neighbor images are not only visually alike and partially correlated but also semantically related. We offer a step-by-step and easy-to-use implementation for the proposed method. Extensive experimentation on several datasets demonstrates that the performance of the proposed method significantly outperforms others.

Innovative Analysis Ready Data(ARD)product and process requirements,software system design,algorithms and implementation at the midstream as necessary-but-notsuffcient precondition of the downstream in a new notion of Space Economy 4.0-Part 1:Problem background in Artificial General Intelligence(AGI)

Aiming at the convergence between Earth observation(EO)Big Data and Artificial General Intelligence(AGI),this two-part paper identifies an innovative,but realistic EO optical sensory imagederived semantics-enriched Analysis Ready Data(ARD)productpair and process gold standard as linchpin for success of a new notion of Space Economy 4.0.To be implemented in operational mode at the space segment and/or midstream segment by both public and private EO big data providers,it is regarded as necessarybut-not-sufficient“horizontal”(enabling)precondition for:(I)Transforming existing EO big raster-based data cubes at the midstream segment,typically affected by the so-called data-rich information-poor syndrome,into a new generation of semanticsenabled EO big raster-based numerical data and vector-based categorical(symbolic,semi-symbolic or subsymbolic)information cube management systems,eligible for semantic content-based image retrieval and semantics-enabled information/knowledge discovery.(II)Boosting the downstream segment in the development of an ever-increasing ensemble of“vertical”(deep and narrow,user-specific and domain-dependent)value–adding information products and services,suitable for a potentially huge worldwide market of institutional and private end-users of space technology.For the sake of readability,this paper consists of two parts.In the present Part 1,first,background notions in the remote sensing metascience domain are critically revised for harmonization across the multidisciplinary domain of cognitive science.In short,keyword“information”is disambiguated into the two complementary notions of quantitative/unequivocal information-as-thing and qualitative/equivocal/inherently ill-posed information-as-data-interpretation.Moreover,buzzword“artificial intelligence”is disambiguated into the two better-constrained notions of Artificial Narrow Intelligence as part-without-inheritance-of AGI.Second,based on a betterdefined and better-understood vocabulary of multidisciplinary terms,existing EO optical sensory image-derived Level 2/ARD products and processes are investigated at the Marr five levels of understanding of an information processing system.To overcome their drawbacks,an innovative,but realistic EO optical sensory image-derived semantics-enriched ARD product-pair and process gold standard is proposed in the subsequent Part 2.

Innovative Analysis Ready Data(ARD)product and process requirements,software system design,algorithms and implementation at the midstream as necessary-but-notsufficient precondition of the downstream in a new notion of Space Economy 4.0-Part 2:Software developments

Aiming at the convergence between Earth observation(EO)Big Data and Artificial General Intelligence(AGI),this paper consists of two parts.In the previous Part 1,existing EO optical sensory imagederived Level 2/Analysis Ready Data(ARD)products and processes are critically compared,to overcome their lack of harmonization/standardization/interoperability and suitability in a new notion of Space Economy 4.0.In the present Part 2,original contributions comprise,at the Marr five levels of system understanding:(1)an innovative,but realistic EO optical sensory image-derived semantics-enriched ARD co-product pair requirements specification.First,in the pursuit of third-level semantic/ontological interoperability,a novel ARD symbolic(categorical and semantic)co-product,known as Scene Classification Map(SCM),adopts an augmented Cloud versus Not-Cloud taxonomy,whose Not-Cloud class legend complies with the standard fully-nested Land Cover Classification System’s Dichotomous Phase taxonomy proposed by the United Nations Food and Agriculture Organization.Second,a novel ARD subsymbolic numerical co-product,specifically,a panchromatic or multispectral EO image whose dimensionless digital numbers are radiometrically calibrated into a physical unit of radiometric measure,ranging from top-of-atmosphere reflectance to surface reflectance and surface albedo values,in a five-stage radiometric correction sequence.(2)An original ARD process requirements specification.(3)An innovative ARD processing system design(architecture),where stepwise SCM generation and stepwise SCM-conditional EO optical image radiometric correction are alternated in sequence.(4)An original modular hierarchical hybrid(combined deductive and inductive)computer vision subsystem design,provided with feedback loops,where software solutions at the Marr two shallowest levels of system understanding,specifically,algorithm and implementation,are selected from the scientific literature,to benefit from their technology readiness level as proof of feasibility,required in addition to proven suitability.To be implemented in operational mode at the space segment and/or midstream segment by both public and private EO big data providers,the proposed EO optical sensory image-derived semantics-enriched ARD product-pair and process reference standard is highlighted as linchpin for success of a new notion of Space Economy 4.0.

Ephemeral gully recognition and accuracy evaluation using deep learning in the hilly and gully region of the Loess Plateau in China

Ephemeral gullies are widely distributed in the hilly and gully region of the Loess Plateau and play a unique role in the slope gully erosion system.Rapid and accurate identification of ephemeral gullies impacts the distribution law and development trend of soil erosion on the Loess Plateau.Deep learning algorithms can quickly and accurately process large data samples that recognize ephemeral gullies from remote sensing images.Here,we investigated ephemeral gullies in the Zhoutungou watershed in the hilly and gully region of the Loess Plateau in China using satellite and unmanned aerial vehicle images and combined a deep learning image semantic segmentation model to realize automatic recognition and feature extraction.Using Accuracy,Precision,Recall,F1value,and AUC,we compared the ephemeral gully recognition results and accuracy evaluation of U-Net,R2U-Net,and SegNet image semantic segmentation models.The SegNet model was ranked first,followed by the R2U-Net and U-Net models,for ephemeral gully recognition in the hilly and gully region of the Loess Plateau.The ephemeral gully length and width between predicted and measured values had RMSE values of 6.78 m and 0.50 m,respectively,indicating that the model has an excellent recognition effect.This study identified a fast and accurate method for ephemeral gully recognition in the hilly and gully region of the Loess Plateau based on remote sensing images to provide an academic reference and practical guidance for soil erosion monitoring and slope and gully management in the Loess Plateau region.