一种基于内容的可扩展性透明Web Cluster体系结构的设计

主要探讨基于内容的Web Cluster，在分析现有方案不足的基础上提出了一种新的基于内容的可扩展性透明Web Cluster体系结构。叙述了其工作机理及所采用的技术方案，重点论述了该集群架构的优越性，最后给出了该集群架构在Linux环境下的模拟实现。

A Throughput-Driven Scheduling Algorithm of Differentiated Service for Web Cluster

Requests distribution is an key technology for Web cluster server. This paper presents a throughput-driven scheduling algorithm （TDSA）. The algorithm adopts the throughput of cluster back-ends to evaluate their load and employs the neural network model to predict the future load so that the scheduling system features a self-learning capability and good adaptability to the change of load. Moreover, it separates static requests from dynamic requests to make full use of the CPU resources and takes the locality of requests into account to improve the cache hit ratio. Experimental re suits from the testing tool of WebBench^TM show better per formance for Web cluster server with TDSA than that with traditional scheduling algorithms.

SSL在Web Cluster中的应用研究与实现

该文深入探讨了各种Web集群架构中部署SSL的解决方案,并详细论述了各解决方案的工作原理、实现技术及其优缺点。针对具体的集群架构,提出了SSLhandoff协议和实现会话重用机制的方案。最后,结合mod_ssl开放源代码给出一个SSL在分布式Layer7型WebCluster中的具体实现。实验证明,该解决方案是可行的、高效的。

PageCluster:一种Web页面层次聚类方法

提出了Web页面聚类算法PageCluster及相应的改进算法ImPageCluster。该方法在兼顾Web站点结构和页面链接的同时,基于各个页面的重要程度对各个超链接进行赋权。与传统聚类算法相比,该算法不需要事先给定相似度阈值。实验结果证实了该算法的可行性和高效性。

A New Self-Adapting Admission Control Algorithm for Differential Service in Web Clusters

A new admission control algorithm considering the network self-similar access characteristics is proposed. Taking advantage of the mathematical model of the network traffic admission control which can effectively overcome the self-similar characteristics of the network requests, through the scheduling of the differential service qucue based on priority while at the same time taking into account various factors including access characteristics of requests, load information, etc, smoothness of the admission control is ensured by the algorithm proposed in this paper. We design a non-linear self-adapting control algorithm by introducing an exponential admission function, thus overcomes the negative aspects introduced by static threshold parameters. Simulation results show that the scheme proposed in this paper can effectively improve the resource utilization of the clusters, while at the same time protecting the service with high priority. Our simulation results also show that this algorithm can improve system stability and reliability too. Key words Web cluster - admission control - differential service - self-similar - self-adapting CLC number TP 393 Foundation item: Supported by the National Natural Science Foundation of China (10375024) and the Hunan Natural Science Foundation of China(03JJY4054)Biography: LIU An-feng(1971-), male, Ph. D candidate, majoring in network computing, Web QoS.

Personalized Metaheuristic Clustering Onto Web Documents

Optimal clustering for the web documents is known to complicated cornbinatorial Optimization problem and it is hard to develop a generally applicable oplimal algorithm. An accelerated simuIated arlneaIing aIgorithm is developed for automatic web document classification. The web document classification problem is addressed as the problem of best describing a match between a web query and a hypothesized web object. The normalized term frequency and inverse document frequency coefficient is used as a measure of the match. Test beds are generated on - line during the search by transforming model web sites. As a result, web sites can be clustered optimally in terms of keyword vectofs of corresponding web documents.

Modeling and Performance Analysis of Dispatching Policies in Dispatcher-Based Web Server Cluster

There are two kinds of dispatching policies in content-aware web server cluster; segregation dispatching policy and mixture dispatching policy. Traditional scheduling algorithms all adopt mixture dispatching policy. They do not consider that dynamic requests＇ serving has the tendency to slow down static requests＇ serving, and that different requests have different resource demands, so they can not use duster＇s resource reasonably and effectively. This paper uses stochastic reward net （SRN） to model and analyze the two dispatching policies, and uses stochastic Petri net package （SPNP） to simulate the models. The simulation results and practical tests both show that segregation dispatching policy is better than mixture dispatching policy. The principle of segregation dispatching policy can guide us to design efficient scheduling algorithm.

A Chinese Web Page Clustering Algorithm Based on the Suffix Tree

In this paper, an improved algorithm, named STC-I, is proposed for Chinese Web page clustering based on Chinese language characteristics, which adopts a new unit choice principle and a novel suffix tree construction policy. The experimental results show that the new algorithm keeps advantages of STC, and is better than STC in precision and speed when they are used to cluster Chinese Web page. Key words clustering - suffix tree - Web mining CLC number TP 311 Foundation item: Supported by the National Information Industry Development Foundation of ChinaBiography: YANG Jian-wu (1973-), male, Ph. D, research direction: information retrieval and text mining.

A Web Server Cluster Solution Based on Twitter Storm

Single Web server would become a bottleneck that influences the availability and stability of Web service. Ten years ago, what had been proposed is to add Web servers for resolving this problem—Web Server Cluster. In recent years, the concept of cloud computing has got rapid development, and is becoming the future development trend of the IT industry. One of the characteristics of cloud computing is putting lots of computing resources together to provide users with a unified service. In this paper, we have proposed a new Cloud-Based Web Server Cluster Solution, based on the existing cloud computing model—Twitter Storm. It involves a new way to handle the web request from client and some other new features compared to the traditional Web Server Cluster. Combining with cloud computing, it would be the new trend of Web Server Cluster, and its feasibility is described in the paper too.

Research of Web Documents Clustering Based on Dynamic Concept

Conceptual clustering is mainly used for solving the deficiency and incompleteness of domain knowledge. Based on conceptual clustering technology and aiming at the institutional framework and characteristic of Web theme information, this paper proposes and implements dynamic conceptual clustering algorithm and merging algorithm for Web documents, and also analyses the super performance of the clustering algorithm in efficiency and clustering accuracy. Key words conceptual clustering - clustering center - dynamic conceptual clustering - theme - web documents clustering CLC number TP 311 Foundation item: Supported by the National “863” Program of China (2002AA111010, 2003AA001032)Biography: WANG Yun-hua(1979-), male, Master candidate, research direction: knowledge engineering and data mining.

DYNAMIC REQUEST DISPATCHING ALGORITHM FOR WEB SERVER CLUSTER

Distributed architectures support increased load on popular web sites by dispatching client requests transparently among multiple servers in a cluster. Packet Single-Rewriting technology and client address hashing algorithm in ONE-IP technology which can ensure application-session-keep have been analyzed, an improved request dispatching algorithm which is simple, effective and supports dynamic load balance has been proposed. In this algorithm, dispatcher evaluates which server node will process request by applying a hash function to the client IP address and comparing the result with its assigned identifier subset; it adjusts the size of the subset according to the performance and current load of each server, so as to utilize all servers' resource effectively. Simulation shows that the improved algorithm has better performance than the original one.

Web Fuzzy Clustering and a Case Study

We combine the web usage mining and fuzzy clustering and give the concept of web fuzzy clustering, and then put forward the web fuzzy clustering processing model which is discussed in detail. Web fuzzy clustering can be used in the web users clustering and web pages clustering. In the end, a case study is given and the result has proved the feasibility of using web fuzzy clustering in web pages clustering. Key words web mining - web usage mining - web fuzzy clustering - WFCM CLC number TP 391 Foundation item: Supported by the National Natural Science Foundation of China (90104005)Biography: LIU Mao-fu (1977-), male, Ph. D candidate, research direction: artificial intelligence, web mining, image mining.

Reliability analysis of web server cluster systems based on proportional hazards model

An approach for web server cluster(WSC)reliability and degradation process analysis is proposed.The reliability process is modeled as a non-homogeneous Markov process(NHMH)composed of several non-homogeneous Poisson processes(NHPPs).The arrival rate of each NHPP corresponds to the system software failure rate which is expressed using Cox s proportional hazards model(PHM)in terms of the cumulative and instantaneous load of the software.The cumulative load refers to software cumulative execution time,and the instantaneous load denotes the rate that the users requests arrive at a server.The result of reliability analysis is a time-varying reliability and degradation process over the WSC lifetime.Finally,the evaluation experiment shows the effectiveness of the proposed approach.

A UNIFIED EXTENDING METHOD FOR CONTENT-IGNORANT WEB PAGE CLUSTERING

The content-ignorant clustering method takes advantages in time complexity and space complexity than the content based methods.In this paper,the authors introduce a unified expanding method for content-ignorant web page clustering by mining the "click-through" log,which tries to solve the problem that the "click-through" log is sparse.The relationship between two nodes which have been expanded is also defined and optimized.Analysis and experiment show that the performance of the new method has improved,by the comparison with the standard content-ignorant method.The new method can also work without iterative clustering.

Incremental Web Usage Mining Based on Active Ant Colony Clustering

To alleviate the scalability problem caused by the increasing Web using and changing users＇ interests, this paper presents a novel Web Usage Mining algorithm-Incremental Web Usage Mining algorithm based on Active Ant Colony Clustering. Firstly, an active movement strategy about direction selection and speed, different with the positive strategy employed by other Ant Colony Clustering algorithms, is proposed to construct an Active Ant Colony Clustering algorithm, which avoid the idle and ＂flying over the plane＂ moving phenomenon, effectively improve the quality and speed of clustering on large dataset. Then a mechanism of decomposing clusters based on above methods is introduced to form new clusters when users＇ interests change. Empirical studies on a real Web dataset show the active ant colony clustering algorithm has better performance than the previous algorithms, and the incremental approach based on the proposed mechanism can efficiently implement incremental Web usage mining.

Fuzzy Clustering Method for Web User Based on Pages Classification

A new method for Web users fuzzy clustering based on analysis of user interest characteristic is proposed in this article. The method first defines page fuzzy categories according to the links on the index page of the site, then computes fuzzy degree of cross page through aggregating on data of Web log. After that, by using fuzzy comprehensive evaluation method, the method constructs user interest vectors according to page viewing times and frequency of hits, and derives the fuzzy similarity matrix from the interest vectors for the Web users. Finally, it gets the clustering result through the fuzzy clustering method. The experimental results show the effectiveness of the method. Key words Web log mining - fuzzy similarity matrix - fuzzy comprehensive evaluation - fuzzy clustering CLC number TP18 - TP311 - TP391 Foundation item: Supported by the Natural Science Foundation of Heilongjiang Province of China (F0304)Biography: ZHAN Li-qiang (1966-), male, Lecturer, Ph. D. research direction: the theory methods of data mining and theory of database.

路径聚类:在Web站点中的知识发现

用户对 Web站点的访问代表了用户对 Web站点上页面的访问兴趣 .这种兴趣程度可以通过用户对 Web站点上页面的浏览顺序表现出来 .在对 Web站点的访问日志进行事务识别后 ,可以根据群体用户对 Web站点的访问顺序进行聚类 ,即路径聚类 ,那么最终每一个聚类集就反映出该聚类集中的全体用户具有相似的访问兴趣 .为了得到这种根据用户访问兴趣而对用户集的划分 ,提出了 K- paths路径聚类方法 .在这种方法中 ,根据用户的访问兴趣定义了新的相似性测量手段和聚类中心 。

Web集群系统负载均衡策略分析与研究

Web集群技术是解决Web服务器系统容量和伸缩能力的重要方法。该文分析了影响Web集群系统性能的主要因素,并提出了一种基于内容的负载均衡算法。该算法以加权负载量来评估服务器负载状态,并通过保证负载局部性来提高cache命中率,从而获得好的负载均衡效果。仿真实验证明,该算法具有较好的适应能力和伸缩性。

Web集群的区分服务与负载均衡策略研究

该文从区分服务和负载均衡两方面研究Web集群系统,设计并实现了一种支持区分服务的负载均衡集群模型,通过系统辨识和资源最优控制,对后端资源统一调度,动态调整每台后端节点不同优先级业务类的资源配额;同时设计了基于业务类的最大空闲节点优先的负载均衡策略以保证各个后端节点同一业务类资源被公平消耗,从而控制端到端延迟,实现比例延迟保证。经实验验证,无论采用轮询调度、最少连接数调度还是最大空闲节点优先的均衡策略,资源最优控制器均能取得良好的区分效果,扩展实验还表明,相对于轮询调度和最少连接数调度,采用最大空闲节点优先的均衡策略,能够使该集群系统在实现比例延迟保证的同时获得最大的性能优化,整体吞吐量提高了25%,平均延迟降低了33%。

Web日志的高效多能挖掘算法

通过对 Web服务器日志文件和客户交易数据进行分析 ,可以发现相似客户群体、相关 Web页面和频繁访问路径 .提出了一种新颖的 Web日志挖掘算法 .在该算法中 ,首先以 Web站点 URL 为行、以 User ID为列建立URL- User ID关联矩阵 ,元素值为用户的访问次数 ,然后 ,对列向量进行相似性分析得到相似客户群体 ,对行向量进行相似性度量获得相关 Web页面 ,对后者再进一步处理还可以发现频繁访问路径 .