Web data mining在远程教育中的应用

采用Web data mining对远程教育进行分析,根据受教育对象存在的个体差异,提出个性化远程学习系统的框架结构思想和个性化服务的理念,对相关信息进行数据挖掘并建立起一个集智能化、个性化为一体的远程教育系统,从而更好地改善远程教育服务的现状。

基于Web挖掘技术的图书馆服务推荐算法分析

为实现精确化、实时化的图书馆图书信息推荐,保证图书馆的服务质量和效率,文中应用Web挖掘技术,设计了一种新型、先进的图书馆服务推荐算法。首先,系统论述了Web挖掘技术的概念、分类及流程。其次,从离线部分、在线部分两个方面入手,实现了图书馆服务推荐系统的框架设计。最后,在参照图书馆服务推荐系统框架的基础上,从建立推荐池、产生初步推荐集、融合离线部分产生个性化推荐集等方面入手,实现了对图书馆服务推荐算法的设计,为相关工作提供了有效的参考。

基于Web-Log Mining的Web文档聚类

速度和效果是聚类算法面临的两大问题.DBSCAN(density based spatial clustering of applications with noise)是典型的基于密度的一种聚类方法,对于大型数据库的聚类实验显示了它在速度上的优越性.提出了一种基于密度的递归聚类算法(recursive density based clustering algorithm,简称RDBC),此算法可以智能地、动态地修改其密度参数.RDBC是基于DBSCAN的一种改进算法,其运算复杂度和DBSCAN相同.通过在Web文档上的聚类实验,结果表明,RDBC不但保留了DBSCAN高速度的优点,而且聚类效果大大优于DBSCAN.

基于Web-Log Mining的N元预测模型

随着Web上用户访问信息的不断增加,特别是Web服务器可提供大量的日志文件,使得有可能对这些大数据集进行知识挖掘,例如,对用户未来的访问进行预测.提出了一种利用服务器日志文件,运用N元(N-gram)预测模型对用户未来可能进行的Web访问请求进行预测.这种模型会选择性地对用户可预测的请求进行预测,从而大大提高了预测精度.实验证明,在自然语言中普遍适用的N元预测模型同样适用于网页预测.同时,采用了一种有效的简化手段,大大压缩了模型的大小,使得5元模型和传统的2元模型大小基本相同,而预测精度提高了1倍.该结果可以广泛地运用到Web上,包括网页的预发送、预取、推荐以及Web上的caching机制.试验是建立在真实的Web日志上的,该算法无论在预测精度上还是在可适用度上都优于以往的算法.

IMPROVING THE INTERESTINGNESS OF WEB USAGE MINING

Improvement on mining the frequently visited groups of web pages was studied. First, in the data preprocessing phrase, we introduce an extra frame filtering step that reduces the negative influence of frame pages on the result page groups. Through recognizing the frame pages in the site documents and constructing the frame subframe relation set, the subframe pages that influence the final mining result can be efficiently filtered. Second, we enhance the mining algorithm with the consideration of both the site topology and the content of the web pages. By the introduction of the normalized content link ratio of the web page and the group interlink degree of the page group, the enhanced algorithm concentrates more on the content pages that are less interlinked together. The experiments show that the new approach can effectively reveal more interesting page groups, which would not be found without these enhancements.

The Study on China’s Flu Prediction Model Based on Web Search Data

Influenza is a kind of infectious disease, which spreads quickly and widely. The outbreak of influenza has brought huge losses to society. In this paper, four major categories of flu keywords, “prevention phase”, “symptom phase”, “treatment phase”, and “commonly-used phrase” were set. Python web crawler was used to obtain relevant influenza data from the National Influenza Center’s influenza surveillance weekly report and Baidu Index. The establishment of support vector regression (SVR), least absolute shrinkage and selection operator (LASSO), convolutional neural networks (CNN) prediction models through machine learning, took into account the seasonal characteristics of the influenza, also established the time series model (ARMA). The results show that, it is feasible to predict influenza based on web search data. Machine learning shows a certain forecast effect in the prediction of influenza based on web search data. In the future, it will have certain reference value in influenza prediction. The ARMA(3,0) model predicts better results and has greater generalization. Finally, the lack of research in this paper and future research directions are given.

A State-of-the-Art Survey on Semantic Web Mining

The integration of the two fast-developing scientific research areas Semantic Web and Web Mining is known as Semantic Web Mining. The huge increase in the amount of Semantic Web data became a perfect target for many researchers to apply Data Mining techniques on it. This paper gives a detailed state-of-the-art survey of on-going research in this new area. It shows the positive effects of Semantic Web Mining, the obstacles faced by researchers and propose number of approaches to deal with the very complex and heterogeneous information and knowledge which are produced by the technologies of Semantic Web.

The design and implementation of web mining in web sites security

The backdoor or information leak of Web servers can be detected by using Web Mining techniques on some abnormal Web log and Web application log data. The security of Web servers can be enhanced and the damage of illegal access can be avoided. Firstly, the system for discovering the patterns of information leakages in CGI scripts from Web log data was proposed. Secondly, those patterns for system administrators to modify their codes and enhance their Web site security were provided. The following aspects were described: one is to combine web application log with web log to extract more information,so web data mining could be used to mine web log for discovering the information that firewall and Information Detection System cannot find. Another approach is to propose an operation module of web site to enhance Web site security. In cluster server session, Density -Based Clustering technique is used to reduce resource cost and obtain better efficiency.

An Effective Network Traffic Data Control Using Improved Apriori Rule Mining

The increasing usage of internet requires a significant system for effective communication. To pro- vide an effective communication for the internet users, based on nature of their queries, shortest routing path is usually preferred for data forwarding. But when more number of data chooses the same path, in that case, bottleneck occurs in the traffic this leads to data loss or provides irrelevant data to the users. In this paper, a Rule Based System using Improved Apriori (RBS-IA) rule mining framework is proposed for effective monitoring of traffic occurrence over the network and control the network traffic. RBS-IA framework integrates both the traffic control and decision making system to enhance the usage of internet trendier. At first, the network traffic data are ana- lyzed and the incoming and outgoing data information is processed using apriori rule mining algorithm. After generating the set of rules, the network traffic condition is analyzed. Based on the traffic conditions, the decision rule framework is introduced which derives and assigns the set of suitable rules to the appropriate states of the network. The decision rule framework improves the effectiveness of network traffic control by updating the traffic condition states for identifying the relevant route path for packet data transmission. Experimental evaluation is conducted by extrac- ting the Dodgers loop sensor data set from UCI repository to detect the effectiveness of theproposed Rule Based System using Improved Apriori (RBS-IA) rule mining framework. Performance evaluation shows that the proposed RBS-IA rule mining framework provides significant improvement in managing the network traffic control scheme. RBS-IA rule mining framework is evaluated over the factors such as accuracy of the decision being obtained, interestingness measure and execution time.

On Structure-based Web Data Extraction: The Model, Method and Application

Web data extraction is to obtain valuable data from the tremendous information resource of the World Wide Web according to the pre - defined pattern. It processes and classifies the data on the Web. Formalization of the procedure of Web data extraction is presented, as well as the description of crawling and extraction algorithm. Based on the formalization, an XML - based page structure description language, TIDL, is brought out, including the object model, the HTML object reference model and definition of tags. At the final part, a Web data gathering and querying application based on Internet agent technology, named Web Integration Services Kit (WISK) is mentioned.

An Efficient Mechanism for Product Data Extraction from E-Commerce Websites

A large amount of data is present on the web which can be used for useful purposes like a product recommendation,price comparison and demand forecasting for a particular product.Websites are designed for human understanding and not for machines.Therefore,to make data machine-readable,it requires techniques to grab data from web pages.Researchers have addressed the problem using two approaches,i.e.,knowledge engineering and machine learning.State of the art knowledge engineering approaches use the structure of documents,visual cues,clustering of attributes of data records and text processing techniques to identify data records on a web page.Machine learning approaches use annotated pages to learn rules.These rules are used to extract data from unseen web pages.The structure of web documents is continuously evolving.Therefore,new techniques are needed to handle the emerging requirements of web data extraction.In this paper,we have presented a novel,simple and efficient technique to extract data from web pages using visual styles and structure of documents.The proposed technique detects Rich Data Region(RDR)using query and correlative words of the query.RDR is then divided into data records using style similarity.Noisy elements are removed using a Common Tag Sequence(CTS)and formatting entropy.The system is implemented using JAVA and runs on the dataset of real-world working websites.The effectiveness of results is evaluated using precision,recall,and F-measure and compared with five existing systems.A comparison of the proposed technique to existing systems has shown encouraging results.

A Parallel Platform for Web Text Mining

With user-generated content, anyone can De a content creator. This phenomenon has infinitely increased the amount of information circulated online, and it is beeoming harder to efficiently obtain required information. In this paper, we describe how natural language processing and text mining can be parallelized using Hadoop and Message Passing Interface. We propose a parallel web text mining platform that processes massive amounts data quickly and efficiently. Our web knowledge service platform is designed to collect information about the IT and telecommunications industries from the web and process this in-formation using natural language processing and data-mining techniques.

Automatic Clustering of User Behaviour Profiles for Web Recommendation System

Web usage mining,content mining,and structure mining comprise the web mining process.Web-Page Recommendation(WPR)development by incor-porating Data Mining Techniques(DMT)did not include end-users with improved performance in the obtainedfiltering results.The cluster user profile-based clustering process is delayed when it has a low precision rate.Markov Chain Monte Carlo-Dynamic Clustering(MC2-DC)is based on the User Behavior Profile(UBP)model group’s similar user behavior on a dynamic update of UBP.The Reversible-Jump Concept(RJC)reviews the history with updated UBP and moves to appropriate clusters.Hamilton’s Filtering Framework(HFF)is designed tofilter user data based on personalised information on automatically updated UBP through the Search Engine(SE).The Hamilton Filtered Regime Switching User Query Probability(HFRSUQP)works forward the updated UBP for easy and accuratefiltering of users’interests and improves WPR.A Probabilistic User Result Feature Ranking based on Gaussian Distribution(PURFR-GD)has been developed to user rank results in a web mining process.PURFR-GD decreases the delay time in the end-to-end workflow for SE personalization in various meth-ods by using the Gaussian Distribution Function(GDF).The theoretical analysis and experiment results of the proposed MC2-DC method automatically increase the updated UBP accuracy by 18.78%.HFRSUQP enabled extensive Maximize Log-Likelihood(ML-L)increases to 15.28%of User Personalized Information Search Retrieval Rate(UPISRT).For feature ranking,the PURFR-GD model defines higher Classification Accuracy(CA)and Precision Ratio(PR)while uti-lising minimum Execution Time(ET).Furthermore,UPISRT's ranking perfor-mance has improved by 20%.

Web数据挖掘技术在电子商务领域中的应用

随着现代科技的不断发展,电子商务作为一种新兴的信息化商业模式,借助互联网的发展迅速占据商业贸易市场上的重要份额。Web数据挖掘技术是一种基于数字化技术,对网络中流传的海量信息进行自动采集、整理、价值发掘的技术。文章从理论阐释入手,先对电子商务的定义、Web数据挖掘技术的定义给予了解析,进而对电子商务系统中该技术的应用模式进行了说明,最后对这一技术在电子商务中应用的作用和意义实施了微观层面的探讨,以期为Web数据挖掘技术更好地在电子商务领域得到应用,实现技术价值的充分发挥提供参考。

基于关联规则的Web日志异常数据挖掘模型

常规Web日志异常数据挖掘模型使用Rough Sets粗集理论挖掘异常日志隐含信息,易受异常数据集的近似分类作用影响,导致挖掘相似度偏低,为此设计基于关联规则的Web日志异常数据挖掘模型。实验结果表明,与对比模型相比,该模型的挖掘相似度较高,性能良好,能够优化Web日志推荐效果。

Web使用模式研究中的数据挖掘

Web使用模式挖掘是利用Web使用数据的高级手段，是对Web使用数据的深层次分析，从而挖掘出有效的、新颖的、潜在的、有用的及最终可以理解的知识，以帮助管理决策。综述了Web使用模式的数据挖掘研究技术的内容、现状和研究的方向。

Web挖掘研究综述

论文介绍了Web挖掘的概念,指出了Web挖掘中存在的问题,给出了Web挖掘研究的三种分类:Web内容挖掘、Web结构挖掘、Web使用挖掘,针对每一种分类介绍了各自的研究对象、表示方法、处理方法、应用领域及最近的研究情况,同时展望了Web挖掘的未来研究方向。

Web使用信息挖掘综述

Web使用信息挖掘可以帮助我们更好地理解Web和Web用户访问模式,这对于开发Web的最大经济潜力是非常关键的。一般来说,Web使用信息挖掘包含三个阶段:数据预处理,模式发现和模式分析。文章以这三个阶段为框架,分别介绍了数据预处理的技术与困难,Web使用信息挖掘中常用的方法和算法,以及主要应用。