SWFP-Miner： an efficient algorithm for mining weighted frequent pattern over data streams

Previous weighted frequent pattern （WFP） mining algorithms are not suitable for data streams for they need multiple database scans. In this paper, we present an efficient algorithm SWFP-Miner to mine weighted frequent pattern over data streams. SWFP-Miner is based on sliding window and can discover important frequent pattern from the recent data. A new refined weight definition is proposed to keep the downward closure property, and two pruning strategies are presented to prune the weighted infrequent pattern. Experimental studies are performed to evaluate the effectiveness and efficiency of SWFP-Miner.

Spatiotemporal deformation characteristics of Outang landslide and identification of triggering factors using data mining

Since the impoundment of Three Gorges Reservoir(TGR)in 2003,numerous slopes have experienced noticeable movement or destabilization owing to reservoir level changes and seasonal rainfall.One case is the Outang landslide,a large-scale and active landslide,on the south bank of the Yangtze River.The latest monitoring data and site investigations available are analyzed to establish spatial and temporal landslide deformation characteristics.Data mining technology,including the two-step clustering and Apriori algorithm,is then used to identify the dominant triggers of landslide movement.In the data mining process,the two-step clustering method clusters the candidate triggers and displacement rate into several groups,and the Apriori algorithm generates correlation criteria for the cause-and-effect.The analysis considers multiple locations of the landslide and incorporates two types of time scales:longterm deformation on a monthly basis and short-term deformation on a daily basis.This analysis shows that the deformations of the Outang landslide are driven by both rainfall and reservoir water while its deformation varies spatiotemporally mainly due to the difference in local responses to hydrological factors.The data mining results reveal different dominant triggering factors depending on the monitoring frequency:the monthly and bi-monthly cumulative rainfall control the monthly deformation,and the 10-d cumulative rainfall and the 5-d cumulative drop of water level in the reservoir dominate the daily deformation of the landslide.It is concluded that the spatiotemporal deformation pattern and data mining rules associated with precipitation and reservoir water level have the potential to be broadly implemented for improving landslide prevention and control in the dam reservoirs and other landslideprone areas.

An Efficient Modelling of Oversampling with Optimal Deep Learning Enabled Anomaly Detection in Streaming Data

Recently,anomaly detection(AD)in streaming data gained significant attention among research communities due to its applicability in finance,business,healthcare,education,etc.The recent developments of deep learning(DL)models find helpful in the detection and classification of anomalies.This article designs an oversampling with an optimal deep learning-based streaming data classification(OS-ODLSDC)model.The aim of the OSODLSDC model is to recognize and classify the presence of anomalies in the streaming data.The proposed OS-ODLSDC model initially undergoes preprocessing step.Since streaming data is unbalanced,support vector machine(SVM)-Synthetic Minority Over-sampling Technique(SVM-SMOTE)is applied for oversampling process.Besides,the OS-ODLSDC model employs bidirectional long short-term memory(Bi LSTM)for AD and classification.Finally,the root means square propagation(RMSProp)optimizer is applied for optimal hyperparameter tuning of the Bi LSTM model.For ensuring the promising performance of the OS-ODLSDC model,a wide-ranging experimental analysis is performed using three benchmark datasets such as CICIDS 2018,KDD-Cup 1999,and NSL-KDD datasets.

Improved Data Stream Clustering Method: Incorporating KD-Tree for Typicality and Eccentricity-Based Approach

Data stream clustering is integral to contemporary big data applications.However,addressing the ongoing influx of data streams efficiently and accurately remains a primary challenge in current research.This paper aims to elevate the efficiency and precision of data stream clustering,leveraging the TEDA(Typicality and Eccentricity Data Analysis)algorithm as a foundation,we introduce improvements by integrating a nearest neighbor search algorithm to enhance both the efficiency and accuracy of the algorithm.The original TEDA algorithm,grounded in the concept of“Typicality and Eccentricity Data Analytics”,represents an evolving and recursive method that requires no prior knowledge.While the algorithm autonomously creates and merges clusters as new data arrives,its efficiency is significantly hindered by the need to traverse all existing clusters upon the arrival of further data.This work presents the NS-TEDA(Neighbor Search Based Typicality and Eccentricity Data Analysis)algorithm by incorporating a KD-Tree(K-Dimensional Tree)algorithm integrated with the Scapegoat Tree.Upon arrival,this ensures that new data points interact solely with clusters in very close proximity.This significantly enhances algorithm efficiency while preventing a single data point from joining too many clusters and mitigating the merging of clusters with high overlap to some extent.We apply the NS-TEDA algorithm to several well-known datasets,comparing its performance with other data stream clustering algorithms and the original TEDA algorithm.The results demonstrate that the proposed algorithm achieves higher accuracy,and its runtime exhibits almost linear dependence on the volume of data,making it more suitable for large-scale data stream analysis research.

A Quarterly High RFM Mining Algorithm for Big Data Management

In today’s highly competitive retail industry,offline stores face increasing pressure on profitability.They hope to improve their ability in shelf management with the help of big data technology.For this,on-shelf availability is an essential indicator of shelf data management and closely relates to customer purchase behavior.RFM(recency,frequency,andmonetary)patternmining is a powerful tool to evaluate the value of customer behavior.However,the existing RFM patternmining algorithms do not consider the quarterly nature of goods,resulting in unreasonable shelf availability and difficulty in profit-making.To solve this problem,we propose a quarterly RFM mining algorithmfor On-shelf products named OS-RFM.Our algorithmmines the high recency,high frequency,and high monetary patterns and considers the period of the on-shelf goods in quarterly units.We conducted experiments using two real datasets for numerical and graphical analysis to prove the algorithm’s effectiveness.Compared with the state-of-the-art RFM mining algorithm,our algorithm can identify more patterns and performs well in terms of precision,recall,and F1-score,with the recall rate nearing 100%.Also,the novel algorithm operates with significantly shorter running times and more stable memory usage than existing mining algorithms.Additionally,we analyze the sales trends of products in different quarters and seasonal variations.The analysis assists businesses in maintaining reasonable on-shelf availability and achieving greater profitability.

A systematic study of Erzhu Erchen decoction against damp-heat internalized type 2 diabetes based on data mining and experimental verification

Background:Erzhu Erchen decoction(EZECD),which is based on Erchen decoction and enhanced with Atractylodes lancea and Atractylodes macrocephala,is widely used for the treatment of dampness and heat(The clinical manifestations of Western medicine include thirst,inability to drink more,diarrhea,yellow urine,red tongue,et al.)internalized disease.Nevertheless,the mechanism of EZECD on damp-heat internalized Type 2 diabetes(T2D)remains unknown.We employed data mining,pharmacology databases and experimental verification to study how EZECD treats damp-heat internalized T2D.Methods:The main compounds or genes of EZECD and damp-heat internalized T2D were obtained from the pharmacology databases.Succeeding,the overlapped targets of EZECD and damp-heat internalized T2D were performed by the Gene Ontology,kyoto encyclopedia of genes and genomes analysis.And the compound-disease targets-pathway network were constructed to obtain the hub compound.Moreover,the hub genes and core related pathways were mined with weighted gene co-expression network analysis based on Gene Expression Omnibus database,the capability of hub compound and genes was valid in AutoDock 1.5.7.Furthermore,and violin plot and gene set enrichment analysis were performed to explore the role of hub genes in damp-heat internalized T2D.Finally,the interactions of hub compound and genes were explored using Comparative Toxicogenomics Database and quantitative polymerase chain reaction.Results:First,herb-compounds-genes-disease network illustrated that the hub compound of EZECD for damp-heat internalized T2D could be quercetin.Consistently,the hub genes were CASP8,CCL2,and AHR according to weighted gene co-expression network analysis.Molecular docking showed that quercetin could bind with the hub genes.Further,gene set enrichment analysis and Gene Ontology represented that CASP8,or CCL2,is negatively involved in insulin secretion response to the TNF or lipopolysaccharide process,and AHR or CCL2 positively regulated lipid and atherosclerosis,and/or including NOD-like receptor signaling pathway,and TNF signaling pathway.Ultimately,the quantitative polymerase chain reaction and western blotting analysis showed that quercetin could down-regulated the mRNA and protein experssion of CASP8,CCL2,and AHR.It was consistent with the results in Comparative Toxicogenomics Database databases.Conclusion:These results demonstrated quercetin could inhibit the expression of CASP8,CCL2,AHR in damp-heat internalized T2D,which improves insulin secretion and inhibits lipid and atherosclerosis,as well as/or including NOD-like receptor signaling pathway,and TNF signaling pathway,suggesting that EZECD may be more effective to treat damp-heat internalized T2D.

Data Mining Based Research of Development Direction of Waist Protection Equipment

[Objectives]To explore the trend of brands towards the design of waist protection products through data mining,and to provide reference for the design concept of the contour of waist protection pillow.[Methods]The structural design information of all waist protection equipment was collected from the national Internet platform,and the data were classified and a database was established.IBM SPSS 26.0 and MATLAB 2018a were used to analyze the data and tabulate them in Tableau 2022.4.After the association rules were clarified,the data were imported into Cinema 4D R21 to create the concept contour of waist protection pillow.[Results]The average and standard deviation of the single airbag design were the highest in all groups,with an average of 0.511 and a standard deviation of 0.502.The average and standard deviation of the upper and lower dual airbags were the lowest in all groups,with an average of 0.015 and a standard deviation of 0.120;the correlation coefficient between single airbag and 120°arc stretching was 0.325,which was positively correlated with each other(P<0.01);the correlation coefficient between multiple airbags and 360°encircling fitting was 0.501,which was positively correlated with each other and had the highest correlation degree(P<0.01).[Conclusions]The single airbag design is well recognized by companies,and has received the highest attention among all brand products.While focusing on single airbag design,most brands will consider the need to add 120°arc stretching elements in product design.At the time of focusing on multiple airbag design,some brands believe that 360°encircling fitting elements need to be added to the product,and the correlation between the two is the highest among all groups.

Forecasting the Academic Performance by Leveraging Educational Data Mining

The study aims to recognize how efficiently Educational DataMining(EDM)integrates into Artificial Intelligence(AI)to develop skills for predicting students’performance.The study used a survey questionnaire and collected data from 300 undergraduate students of Al Neelain University.The first step’s initial population placements were created using Particle Swarm Optimization(PSO).Then,using adaptive feature space search,Educational Grey Wolf Optimization(EGWO)was employed to choose the optimal attribute combination.The second stage uses the SVMclassifier to forecast classification accuracy.Different classifiers were utilized to evaluate the performance of students.According to the results,it was revealed that AI could forecast the final grades of students with an accuracy rate of 97%on the test dataset.Furthermore,the present study showed that successful students could be selected by the Decision Tree model with an efficiency rate of 87.50%and could be categorized as having equal information ratio gain after the semester.While the random forest provided an accuracy of 28%.These findings indicate the higher accuracy rate in the results when these models were implemented on the data set which provides significantly accurate results as compared to a linear regression model with accuracy(12%).The study concluded that the methodology used in this study can prove to be helpful for students and teachers in upgrading academic performance,reducing chances of failure,and taking appropriate steps at the right time to raise the standards of education.The study also motivates academics to assess and discover EDM at several other universities.

Integrated data mining and network pharmacology to discover a novel traditional Chinese medicine prescription against diabetic retinopathy and reveal its mechanism

Background:Diabetic retinopathy(DR)is currently the leading cause of blindness in elderly individuals with diabetes.Traditional Chinese medicine(TCM)prescriptions have shown remarkable effectiveness for treating DR.This study aimed to screen a novel TCM prescription against DR from patents and elucidate its medication rule and molecular mechanism using data mining,network pharmacology,molecular docking and molecular dynamics(MD)simulation.Method:TCM prescriptions for treating DR was collected from patents and a novel TCM prescription was identified using data mining.Subsequently,the mechanism of the novel TCM prescription against DR was explored by constructing a network of core TCMs-core active ingredients-core targets-core pathways.Finally,molecular docking and MD simulation were employed to validate the findings from network pharmacology.Result:The TCMs of the collected prescriptions primarily possessed bitter and cold properties with heat-clearing and supplementing effects,attributed to the liver,lung and kidney channels.Notably,a novel TCM prescription for treating DR was identified,composed of Lycii Fructus,Chrysanthemi Flos,Astragali Radix and Angelicae Sinensis Radix.Twenty core active ingredients and ten core targets of the novel TCM prescription for treating DR were screened.Moreover,the novel TCM prescription played a crucial role for treating DR by inhibiting inflammatory response,oxidative stress,retinal pigment epithelium cell apoptosis and retinal neovascularization through various pathways,such as the AGE-RAGE signaling pathway in diabetic complications and the MAPK signaling pathway.Finally,molecular docking and MD simulation demonstrated that almost all core active ingredients exhibited satisfactory binding energies to core targets.Conclusions:This study identified a novel TCM prescription and unveiled its multi-component,multi-target and multi-pathway characteristics for treating DR.These findings provide a scientific basis and novel insights into the development of drugs for DR prevention and treatment.

Detection of Knowledge on Social Media Using Data Mining Techniques

In light of the rapid growth and development of social media, it has become the focus of interest in many different scientific fields. They seek to extract useful information from it, and this is called (knowledge), such as extracting information related to people’s behaviors and interactions to analyze feelings or understand the behavior of users or groups, and many others. This extracted knowledge has a very important role in decision-making, creating and improving marketing objectives and competitive advantage, monitoring events, whether political or economic, and development in all fields. Therefore, to extract this knowledge, we need to analyze the vast amount of data found within social media using the most popular data mining techniques and applications related to social media sites.

Aviation Safety and Data Mining in Marketing Dimension

The aviation industry is a sector that is developing, changing and growing every day in terms of technological and legal framework. There are generally three factors that enable airlines to hold on to the market. These factors are safety, service quality and price. Airline companies can analyze the customers in the market with a focus on price and quality and develop a business model according to their expectations. For example, business class and economy class passenger expectations are different from each other, so the service and price to be offered to them will be different. However, all customers have one common expectation and that is safety. No matter how high quality the service is or how cheap the price is, no one wants to fly with an airline or plane that is not safe. From an airline company’s point of view, an accident or breakdown of one of the company’s aircraft can cause irreparable image loss and financial damage. If we look at past examples, we see that there are many airline companies or maintenance organizations that could not recover after an accident and went bankrupt. Safety is an indispensable factor. Therefore, there is a unit in the sector called the safety management system (SMS), which collects data by taking a proactive and reactive approach. The way and purpose of the safety management system is to take a proactive approach to recognize and prevent unsafe situations before they cause accidents or breakdowns, or to take a reactive approach to find the causes of accidents and breakdowns that have occurred as a result of certain factors and to take the necessary measures to prevent the same situations from happening again in the sector. The field of data mining, which is necessary to predict the future behavior of customers in the field of marketing, is an area that marketing also values. In this study, data mining studies to ensure safety in the aviation industry and the security of customer information in marketing will be emphasized, firstly, the concept and importance of data mining will be mentioned.

Construction and Practice of Teaching Evaluation System Based on Data Mining

The teaching quality evaluation system based on data mining technology can accurately and fairly identify the core driving factors to improve teaching quality.This method adopts the analysis of big data correlation rules,including data collection and processing preparation steps,builds the data warehouse of association rules,and then generates an educational quality evaluation framework using the principle of data mining.Based on this,this paper analyzes the construction design and method of the teaching evaluation system under data mining,hoping to provide help for the improvement of the teaching evaluation system and the improvement of teaching quality.

Exploring the medication pattern and mechanism of action of traditional Chinese medicine in treating polycystic ovary syndrome with kidney deficiency and blood stasis based on data mining and network pharmacology

Background:Using network pharmacology to explore the potential molecular mechanism of traditional Chinese medicine in treating polycystic ovary syndrome(PCOS)with kidney deficiency and blood stasis syndrome.Method:Collect the related literature materials of PCOS with kidney deficiency and blood stasis syndrome treated by traditional Chinese medicine in four databases in recent ten years,extract the information of prescriptions and complete the frequency analysis.Traditional Chinese Medicine Systems Pharmacology Database was used to screen out the effective components.Use Online Mendelian Inheritance in Man and other databases to screen PCOS disease targets.The intersection targets obtained by clustering prescription and PCOS disease targets were submitted to STRING database for protein-protein interaction network analysis,and Gene Ontology(GO)and Kyoto Encyclopedia of Genes and Genomes pathways were analysed by Metascape.Result:There are 155 kinds of traditional Chinese medicines used in the literature.The most commonly utilized ones are Cuscutae Semen,Angelicae Sinensis Radix,and Rehmanniae Radix Praeparata.The results of the cluster analysis indicated that the plants most commonly found throughout the prescription were Leonuri Herba,Lycopi Herba,Dipsaci Radix,etc.GO results show that biological processes include cell reaction to organic nitrogen compounds and cell reaction to nitrogen compounds.The functional display of GO molecule includes cytokine receptor binding,signal receptor regulator activity and so on.Kyoto Encyclopedia of Genes and Genomes results show that the possible mechanisms of action are cancer pathway,an endocrine resistance signal pathway.Conclusion:Through data mining,the cluster prescription for PCOS with kidney deficiency and blood stasis syndrome is Leonuri Herba,Lycopi Herba,Dipsaci Radix,etc.The network pharmacology research of cluster prescription shows that the main drug components for treating PCOS with kidney deficiency and blood stasis syndrome are quercetin,kaempferol,luteolin,tanshinone IIA,etc.,which act on PTGS2,NCOA2,and other targets,and treat PCOS with kidney deficiency and blood stasis syndrome through cancer and endocrine resistance.

An Efficient Outlier Detection Approach on Weighted Data Stream Based on Minimal Rare Pattern Mining

The distance-based outlier detection method detects the implied outliers by calculating the distance of the points in the dataset, but the computational complexity is particularly high when processing multidimensional datasets. In addition, the traditional outlier detection method does not consider the frequency of subsets occurrence, thus, the detected outliers do not fit the definition of outliers (i.e., rarely appearing). The pattern mining-based outlier detection approaches have solved this problem, but the importance of each pattern is not taken into account in outlier detection process, so the detected outliers cannot truly reflect some actual situation. Aimed at these problems, a two-phase minimal weighted rare pattern mining-based outlier detection approach, called MWRPM-Outlier, is proposed to effectively detect outliers on the weight data stream. In particular, a method called MWRPM is proposed in the pattern mining phase to fast mine the minimal weighted rare patterns, and then two deviation factors are defined in outlier detection phase to measure the abnormal degree of each transaction on the weight data stream. Experimental results show that the proposed MWRPM-Outlier approach has excellent performance in outlier detection and MWRPM approach outperforms in weighted rare pattern mining.

Disaster prediction of coal mine gas based on data mining

The technique of data mining was provided to predict gas disaster in view of the characteristics of coal mine gas disaster and feature knowledge based on gas disaster. The rough set theory was used to establish data mining model of gas disaster prediction, and rough set attributes relations was discussed in prediction model of gas disaster to supplement the shortages of rough intensive reduction method by using information en- tropy criteria.The effectiveness and practicality of data mining technology in the prediction of gas disaster is confirmed through practical application.

Incremental Learning Framework for Mining Big Data Stream

At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data,it also supposes that all data are pre-classified.Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features.The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model.The proposed model is implemented in spark architecture.For each data stream,the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation.This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency.The results show that the proposed model outperforms stateof-the-art performance in terms of accuracy(0.861)precision(0.9328)and minimal MSE(0.0416).

Anomalous Network Packet Detection Using Data Stream Mining

In recent years, significant research has been devoted to the development of Intrusion Detection Systems (IDS) able to detect anomalous computer network traffic indicative of malicious activity. While signature-based IDS have proven effective in discovering known attacks, anomaly-based IDS hold the even greater promise of being able to automatically detect previously undocumented threats. Traditional IDS are generally trained in batch mode, and therefore cannot adapt to evolving network data streams in real time. To resolve this limitation, data stream mining techniques can be utilized to create a new type of IDS able to dynamically model a stream of network traffic. In this paper, we present two methods for anomalous network packet detection based on the data stream mining paradigm. The first of these is an adapted version of the DenStream algorithm for stream clustering specifically tailored to evaluate network traffic. In this algorithm, individual packets are treated as points and are flagged as normal or abnormal based on their belonging to either normal or outlier clusters. The second algorithm utilizes a histogram to create a model of the evolving network traffic to which incoming traffic can be compared using Pearson correlation. Both of these algorithms were tested using the first week of data from the DARPA ’99 dataset with Generic HTTP, Shell-code and Polymorphic attacks inserted. We were able to achieve reasonably high detection rates with moderately low false positive percentages for different types of attacks, though detection rates varied between the two algorithms. Overall, the histogram-based detection algorithm achieved slightly superior results, but required more parameters than the clustering-based algorithm. As a result of its fewer parameter requirements, the clustering approach can be more easily generalized to different types of network traffic streams.

Research on Forecast Technologyof Mine Gas Emission Based onFuzzy Data Mining(FDM)

The safe production of coalmine can be further improved by forecasting the quantity of gas emission based on the real-time data and historical data which the gas monitoring system has saved. By making use of the advantages of data warehouse and data mining technology for processing large quantity of redundancy data, the method and its application of forecasting mine gas emission quantity based on FDM were studied. The constructing fuzzy resembling relation and clustering analysis were proposed, which the potential relationship inside the gas emission data may be found. The mode finds model and forecast model were presented, and the detailed approach to realize this forecast was also proposed, which have been applied to forecast the gas emission quantity efficiently.

Data mining and well logging interpretation: application to a conglomerate reservoir

Data mining is the process of extracting implicit but potentially useful information from incomplete, noisy, and fuzzy data. Data mining offers excellent nonlinear modeling and self-organized learning, and it can play a vital role in the interpretation of well logging data of complex reservoirs. We used data mining to identify the lithologies in a complex reservoir. The reservoir lithologies served as the classification task target and were identified using feature extraction, feature selection, and modeling of data streams. We used independent component analysis to extract information from well curves. We then used the branch-and- bound algorithm to look for the optimal feature subsets and eliminate redundant information. Finally, we used the C5.0 decision-tree algorithm to set up disaggregated models of the well logging curves. The modeling and actual logging data were in good agreement, showing the usefulness of data mining methods in complex reservoirs.

Clustering algorithm for multiple data streams based on spectral component similarity

A new algorithm for clustering multiple data streams is proposed.The algorithm can effectively cluster data streams which show similar behavior with some unknown time delays.The algorithm uses the autoregressive （AR） modeling technique to measure correlations between data streams.It exploits estimated frequencies spectra to extract the essential features of streams.Each stream is represented as the sum of spectral components and the correlation is measured component-wise.Each spectral component is described by four parameters,namely,amplitude,phase,damping rate and frequency.The ε-lag-correlation between two spectral components is calculated.The algorithm uses such information as similarity measures in clustering data streams.Based on a sliding window model,the algorithm can continuously report the most recent clustering results and adjust the number of clusters.Experiments on real and synthetic streams show that the proposed clustering method has a higher speed and clustering quality than other similar methods.