Research on the Big Data Cloud Computing Based on the Network Data Mining

The big data cloud computing is a new computing mode,which integrates the distributed processing,the parallel processing,the network computing,the virtualization technology,the load balancing and other network technologies.Under the operation of the big data cloud computing system,the computing resources can be distributed in a resource pool composed of a large number of the computers,allowing users to connect with the remote computer systems according to their own data information needs.

Research on the big data feature mining technology based on the cloud computing

The cloud computing platform has the functions of efficiently allocating the dynamic resources, generating the dynamic computing and storage according to the user requests, and providing the good platform for the big data feature analysis and mining. The big data feature mining in the cloud computing environment is an effective method for the elficient application of the massive data in the information age. In the process of the big data mining, the method o f the big data feature mining based on the gradient sampling has the poor logicality. It only mines the big data features from a single-level perspective, which reduces the precision of the big data feature mining.

Research on the Distributed Data Mining Cloud Framework Oriented Internet of Things

The rapid development of Internet of Things imposes new requirements on the data mining system, due to the weak capability of traditional distributed networking data mining. To meet the needs of the Internet of Things, this paper proposes a novel distributed data-mining model to realize the seamless access between cloud computing and distributed data mining. The model is based on the cloud computing architecture, which belongs to the type of incredible nodes.

Data Mining in Electronic Commerce: Benefits and Challenges

Huge volume of structured and unstructured data which is called big data, nowadays, provides opportunities for companies especially those that use electronic commerce (e-commerce). The data is collected from customer’s internal processes, vendors, markets and business environment. This paper presents a data mining (DM) process for e-commerce including the three common algorithms: association, clustering and prediction. It also highlights some of the benefits of DM to e-commerce companies in terms of merchandise planning, sale forecasting, basket analysis, customer relationship management and market segmentation which can be achieved with the three data mining algorithms. The main aim of this paper is to review the application of data mining in e-commerce by focusing on structured and unstructured data collected thorough various resources and cloud computing services in order to justify the importance of data mining. Moreover, this study evaluates certain challenges of data mining like spider identification, data transformations and making data model comprehensible to business users. Other challenges which are supporting the slow changing dimensions of data, making the data transformation and model building accessible to business users are also evaluated. A clear guide to e-commerce companies sitting on huge volume of data to easily manipulate the data for business improvement which in return will place them highly competitive among their competitors is also provided in this paper.

MobSafe:Cloud Computing Based Forensic Analysis for Massive Mobile Applications Using Data Mining

With the explosive increase in mobile apps, more and more threats migrate from traditional PC client to mobile device. Compared with traditional Win＋Intel alliance in PC, Android＋ARM alliance dominates in Mobile Internet, the apps replace the PC client software as the major target of malicious usage. In this paper, to improve the security status of current mobile apps, we propose a methodology to evaluate mobile apps based on cloud computing platform and data mining. We also present a prototype system named MobSafe to identify the mobile app＇s virulence or benignancy. Compared with traditional method, such as permission pattern based method, MobSafe combines the dynamic and static analysis methods to comprehensively evaluate an Android app. In the implementation, we adopt Android Security Evaluation Framework （ASEF） and Static Android Analysis Framework （SAAF）, the two representative dynamic and static analysis methods, to evaluate the Android apps and estimate the total time needed to evaluate all the apps stored in one mobile app market. Based on the real trace from a commercial mobile app market called AppChina, we can collect the statistics of the number of active Android apps, the average number apps installed in one Android device, and the expanding ratio of mobile apps. As mobile app market serves as the main line of defence against mobile malwares, our evaluation results show that it is practical to use cloud computing platform and data mining to verify all stored apps routinely to filter out malware apps from mobile app markets. As the future work, MobSafe can extensively use machine learning to conduct automotive forensic analysis of mobile apps based on the generated multifaceted data in this stage.

PDMiner平台的主动配电网安全监测系统

主动配电网系统结构复杂、DG渗透率高、负荷波动大、控制要求严格,运行数据信息呈现出大量、高速、多样和价值的特点。针对主动配电网大数据处理的问题,依据云计算理论,采用数据挖掘技术和分布并行计算方式,建立了一套PDMiner大数据挖掘平台的主动配电网安全监测系统模型。分别研究了系统对历史数据和实时故障数据的处理。研究表明该系统通过对历史数据的处理可进行风险区域划分、制定规划方案、运行状态评价,通过对故障数据的处理可进行故障分析与处理,并为配电网安全改造提供依据。通过仿真算例验证了该系统的工作效果。

基于雾计算的电力信息高效存储与共享算法

针对电力信息数据量较大且存储速度较慢的问题,提出了一种基于雾计算的电力信息存储与共享方法。该方法包含了基于差分同步的数据存储方式与基于雾计算的组合数据共享机制。数据存储方法将一部分数据的计算和存储工作转移到雾服务器中,以减少云计算平台的工作负载,从而实现更高效的数据存储;组合数据共享机制通过在雾端建立虚拟化数据共享机制来防止电力信息数据的丢失、损坏及恶意修改。仿真实验与结果表明,该方法能显著减少云服务器的响应次数,进而减轻云计算的负担,具有更高的数据恢复率,相比于传统方法,所提方法可实现高效、安全的电力信息存储及共享。

智能矿山综合管控平台研究及应用

为提高矿山安全生产能力和治理水平,建成安全、高效、绿色、智能矿山。以山东某矿企业智能矿山建设为背景,依托云计算、大数据、物联网、人工智能等技术,提出一套智能化综合管控平台建设方案。实现矿山安全生产控制可视、指挥调度、实时监测预警、追溯定位、系统联动、趋势研判、安全防范、决策支持等功能。经初步试验验证,平台稳定可靠,有效提升矿山的经济效益、安全效益和管理效益。

云计算环境下的数据挖掘技术研究

本文探讨了云计算环境下数据挖掘技术的发展及面临的挑战,重点分析了云计算技术的基础、数据挖掘在各领域的应用和云计算环境下数据挖掘的优势与挑战,强调了深度学习、大数据技术、云计算基础设施的支持及隐私保护的重要性。本研究设计了基于云计算平台的数据挖掘框架方案,并提出有效的数据安全与隐私保护机制,旨在探索数据挖掘技术在云计算环境中的有效应用。

基于云计算的数据挖掘聚类算法研究

近年来大数据及云计算技术、人工智能技术的发展,使得K-Means聚类算法、DBSCAN聚类算法、BIRCH聚类算法、Cluster数据分布算法不断涌现,但不同算法在面对海量化、多样化网络数据样本时的性能存在差异。基于此,从不同数据文本关联性、数据集资源的海量性角度出发,利用云计算Spark分布式架构、HDFS(Hadoop Distributed FileSystem)分布式文件系统、Spark SQL数据计算引擎、YARN(Yet Another Resource Negotiator)资源管理器等软件,建构起融合K-Means聚类算法、BIRCH(Balanced Iterative Reducing and Clustering using Hierarchies)聚类算法的数据挖掘模型,根据CF树聚类的判别函数确定被测试数据集的类别,由Spark计算模型将完成聚类的数据集分布式缓存至网络节点内存中,以实现对海量化网络数据的挖掘、聚类及存储操作。

继电保护系统中的数据挖掘与决策分析

阐述云计算和大数据技术在电力系统中的关键作用,通过挖掘继电保护事件数据,分析数据挖掘技术在提高响应效率方面的应用。介绍智能决策系统的原理,探讨基于云计算与大数据的智能应用。

基于大数据的虚实融合教学服务平台设计

文章探究了基于大数据的虚实融合教学服务平台设计,阐述了平台的总体设计方式,详解了教育平台功能。为提高平台大数据系统的应用能力,文章研究了云计算资源算法,描述了云计算中的问题,提高了云计算的合并应用效率,并且厘清了虚拟机的动态调度步骤,提高了教学平台系统的运行效率。实验结果表明,基于大数据的虚实融合教学服务平台设计能够有效提高学生的学习成绩。

MBGM: A Graph-Mining Tool Based on MapReduce and BSP

This paper proposes an analytical mining tool for big graph data based on MapReduce and bulk synchronous parallel （BSP） com puting model. The tool is named Mapreduce and BSP based Graphmining tool （MBGM）. The core of this mining system are four sets of parallel graphmining algorithms programmed in the BSP parallel model and one set of data extractiontransformationload ing （ETE） algorithms implemented in MapReduce. To invoke these algorithm sets, we designed a workflow engine which optimized for cloud computing. Finally, a welldesigned data management function enables users to view, delete and input data in the Ha doop distributed file system （HDFS）. Experiments on artificial data show that the components of graphmining algorithm in MBGM are efficient.

Parallelized User Clicks Recognition from Massive HTTP Data Based on Dependency Graph Model

With increasingly complex website structure and continuously advancing web technologies,accurate user clicks recognition from massive HTTP data,which is critical for web usage mining,becomes more difficult.In this paper,we propose a dependency graph model to describe the relationships between web requests.Based on this model,we design and implement a heuristic parallel algorithm to distinguish user clicks with the assistance of cloud computing technology.We evaluate the proposed algorithm with real massive data.The size of the dataset collected from a mobile core network is 228.7GB.It covers more than three million users.The experiment results demonstrate that the proposed algorithm can achieve higher accuracy than previous methods.

Metaheuristics with Machine Learning Enabled Information Security on Cloud Environment

The increasing quantity of sensitive and personal data being gathered by data controllers has raised the security needs in the cloud environment.Cloud computing(CC)is used for storing as well as processing data.Therefore,security becomes important as the CC handles massive quantity of outsourced,and unprotected sensitive data for public access.This study introduces a novel chaotic chimp optimization with machine learning enabled information security(CCOML-IS)technique on cloud environment.The proposed CCOML-IS technique aims to accomplish maximum security in the CC environment by the identification of intrusions or anomalies in the network.The proposed CCOML-IS technique primarily normalizes the networking data by the use of data conversion and min-max normalization.Followed by,the CCOML-IS technique derives a feature selection technique using chaotic chimp optimization algorithm(CCOA).In addition,kernel ridge regression(KRR)classifier is used for the detection of security issues in the network.The design of CCOA technique assists in choosing optimal features and thereby boost the classification performance.A wide set of experimentations were carried out on benchmark datasets and the results are assessed under several measures.The comparison study reported the enhanced outcomes of the CCOML-IS technique over the recent approaches interms of several measures.

基于改进Apriori算法的大数据AR挖掘仿真

针对传统方法存在的数据挖掘结果准确率不高和数据处理平台响应时间较长的问题,提出基于改进Apriori算法的大数据关联规则挖掘方法。建立一个基于云计算技术的数据处理平台,平台以基础设施层和数据信息层的数据采集与数据存储为基础,然后借助业务逻辑层实现数据清洗、挖掘以及分析,数据清洗过程可实现数据降噪与标准化处理。根据数据处理平台的数据处理结果,利用Map Reduce编程模型优化Apriori数据挖掘算法,对清洗后的数据进行关联规则挖掘,提升数据关联规则挖掘的效率。最后再通过设计的数据处理平台利用应用接口连接业务逻辑层与服务层,并通过展示层向用户层提供数据分析结果。以旅游数据为例进行仿真分析,结果表明该算法能够为用户提供准确的数据,并且平台的响应速度较快。

云计算技术在计算机大数据分析中的应用探析

随着信息技术的不断发展,计算机所存数据种类日渐丰富,计算机大数据技术成为未来发展的一个主流方向。主要探讨了云计算技术在计算机大数据分析中的应用,以期解决大数据应用中的核心问题,更加高效地进行数据处理。

基于云计算下现代生态农业物联网监控系统的设计

近年来,随着物联网技术在农业生产中的广泛应用与发展,如何高性能地存储农业物联网数据,保证农业物联网系统的安全性是现代生态农业物联网系统设计的关键问题。为此,基于新一代信息技术,融合物联网、云计算、数据挖掘等技术,形成一个基于云计算的现代生态农业物联网监控系统平台,并提出其在现代农业领域应用的具体实施方案。通过搭建的测试环境对系统的应用功能及工作性能进行测试,结果表明:设计的基于云计算下现代生态农业物联网监控系统可以实现对农业生产区域及各个生产基地环境信息的实时监测,为用户提供了良好的功能体验及性能表现,在降低系统开发成本和确保其可靠性及安全性方面发挥了重要作用。

基于数字孪生的煤矿智能管控平台架构研究与实现

为完成煤矿智能化升级转型,对煤矿智能开采现状进行了分析,提出基于数字孪生的煤矿智能管控平台的概念。该平台以数字孪生技术为核心,结合GIS、建模、仿真等关键技术,融合应用物联网、大数据、云计算及人工智能,赋予数据空间维度信息,使数字矿山与物理矿山之间进行实时交互,忠实地复现物理矿山整个生命周期的运行状态与轨迹,推动煤矿传统生产全面升级,向数字化、智能化转型。实际应用测试表明,该平台提高了煤矿数据采集、存储、传输和分析的能力,降低了安全事故发生概率,能有效为数字孪生和智能管控煤矿生产各个环节的深度融合提供服务。

基于复杂地质环境条件下的TBM施工数据挖掘与云计算综合管理平台搭建

针对目前大型地下工程在复杂地质环境条件下硬岩隧道掘进机(tunnel boring machine,TBM)施工的安全保障工作存在“缺数据、缺平台、缺分析”以及海量数据难以挖掘与计算分析的问题,提出了TBM施工数据“Born by digit,Born in format,Born to the cloud”的理念,研究构建了基于云计算和大数据挖掘技术的复杂地质环境状态TBM施工信息综合管理平台,该平台通过环境地质分析以及数据采集、存储和计算分析,建立基于TBM锚杆钻机随钻测量系统和岩体渣片图像识别系统实时采集岩体数字化信息,并实时采集TBM施工过程中PLC(可编程逻辑控制器)数据,建立Mongo DB TMB多源施工信息大数据仓库,同时基于Openstack技术搭建私有云计算平台以及在SaaS层部署算法组件开展TBM大数据挖掘。该平台的搭建不仅能提供基础信息,帮助提升安全管控水平,更重要的是可为TBM在复杂地质环境条件下对于各类瓶颈问题的研究解决提供数据挖掘和计算分析的支撑,并通过其所建立的知识库和规则库指导TBM设计、选型、控制。该研究成果在中国吉林引松供水工程项目上得到应用,并取得了较好的效益。