A new data normalization method for unsupervised anomaly intrusion detection

Unsupervised anomaly detection can detect attacks without the need for clean or labeled training data.This paper studies the application of clustering to unsupervised anomaly detection(ACUAD).Data records are mapped to a feature space.Anomalies are detected by determining which points lie in the sparse regions of the feature space.A critical element for this method to be effective is the definition of the distance function between data records.We propose a unified normalization distance framework for records with numeric and nominal features mixed data.A heuristic method that computes the distance for nominal features is proposed,taking advantage of an important characteristic of nominal features-their probability distribution.Then,robust methods are proposed for mapping numeric features and computing their distance,these being able to tolerate the impact of the value difference in scale and diversification among features,and outliers introduced by intrusions.Empirical experiments with the KDD 1999 dataset showed that ACUAD can detect intrusions with relatively low false alarm rates compared with other approaches.

Unsupervised object detection with scene-adaptive concept learning

Object detection is one of the hottest research directions in computer vision,has already made impressive progress in academia,and has many valuable applications in the industry.However,the mainstream detection methods still have two shortcomings:(1)even a model that is well trained using large amounts of data still cannot generally be used across different kinds of scenes;(2)once a model is deployed,it cannot autonomously evolve along with the accumulated unlabeled scene data.To address these problems,and inspired by visual knowledge theory,we propose a novel scene-adaptive evolution unsupervised video object detection algorithm that can decrease the impact of scene changes through the concept of object groups.We first extract a large number of object proposals from unlabeled data through a pre-trained detection model.Second,we build the visual knowledge dictionary of object concepts by clustering the proposals,in which each cluster center represents an object prototype.Third,we look into the relations between different clusters and the object information of different groups,and propose a graph-based group information propagation strategy to determine the category of an object concept,which can effectively distinguish positive and negative proposals.With these pseudo labels,we can easily fine-tune the pretrained model.The effectiveness of the proposed method is verified by performing different experiments,and the significant improvements are achieved.

An unsupervised heterogeneous change detection method based on image translation network and post-processing algorithm

The change detection(CD)of heterogeneous remote sensing images is an important but challenging task.The difficulty is to obtain the change information by directly comparing the different statistical characteristics of the images acquired by different sensors.This paper proposes an unsupervised method for heterogeneous image CD based on an image domain transfer network.First,an attention mechanism is added to the Cycle-generative adversarial networks(Cycle-GANs)to obtain a more consistent feature expression by transferring bi-temporal heterogeneous images to the common domain.The Euclidean distance of the corresponding pixels is calculated in the common domain to form a difference map,and a threshold algorithm is applied to get a rough change map.Finally,the proposed adaptive Discrete Cosine Transform(DCT)algorithm reduces the noise introduced by false detection,and the final change map is obtained.The proposed method is verified on three real heterogeneous CD datasets and compared with the current state-of-the-art methods.The results show that the proposed method is accurate and robust for performing heterogeneous CD tasks.

Unsupervised machine learning to classify crystal structures according totheir structural distortion: A case study on Li-argyrodite solid-state electrolytes

High-throughput approaches in computational materials discovery often yield a combinatorial explosionthat makes the exhaustive rendering of complete structural and chemical spaces impractical. A commonbottleneck when screening new compounds with archetypal crystal structures is the lack of fast and reliabledecision-making schemes to quantitatively classify the computed candidates as inliers or outliers (too distortedstructures). Machine learning-aided workflows can solve this problem and make geometrical optimizationprocedures more efficient. However, for this to occur, there is still a lack of appropriate combinations ofsuitable geometrical descriptors and accurate unsupervised models which are capable of accurately differentiating between systems with subtle structural changes. Here, considering as a case study the compositionalscreening of cubic Li-argyrodites solid electrolytes, we tackle this problem head on. We find that Steinhardtorder parameters are very accurate descriptors of the cubic argyrodite structure to train a range of commonunsupervised outlier detection models. And, most importantly, the approach enables us to automatically classifycrystal structures with uncertainty control. The resulting models can then be used to screen computed structureswith respect to an user-defined error threshold and discard too distorted structures during geometricaloptimization procedures. Implemented as a decision node in computer-aided materials discovery workflows,this approach can be employed to perform autonomous high-throughput screening methods and make the useof computational and data storage resources more efficient.