Modeling of Optimal Deep Learning Based Flood Forecasting Model Using Twitter Data

Aflood is a significant damaging natural calamity that causes loss of life and property.Earlier work on the construction offlood prediction models intended to reduce risks,suggest policies,reduce mortality,and limit property damage caused byfloods.The massive amount of data generated by social media platforms such as Twitter opens the door toflood analysis.Because of the real-time nature of Twitter data,some government agencies and authorities have used it to track natural catastrophe events in order to build a more rapid rescue strategy.However,due to the shorter duration of Tweets,it is difficult to construct a perfect prediction model for determiningflood.Machine learning(ML)and deep learning(DL)approaches can be used to statistically developflood prediction models.At the same time,the vast amount of Tweets necessitates the use of a big data analytics(BDA)tool forflood prediction.In this regard,this work provides an optimal deep learning-basedflood forecasting model with big data analytics(ODLFF-BDA)based on Twitter data.The suggested ODLFF-BDA technique intends to anticipate the existence offloods using tweets in a big data setting.The ODLFF-BDA technique comprises data pre-processing to convert the input tweets into a usable format.In addition,a Bidirectional Encoder Representations from Transformers(BERT)model is used to generate emotive contextual embed-ding from tweets.Furthermore,a gated recurrent unit(GRU)with a Multilayer Convolutional Neural Network(MLCNN)is used to extract local data and predict theflood.Finally,an Equilibrium Optimizer(EO)is used tofine-tune the hyper-parameters of the GRU and MLCNN models in order to increase prediction performance.The memory usage is pull down lesser than 3.5 MB,if its compared with the other algorithm techniques.The ODLFF-BDA technique’s performance was validated using a benchmark Kaggle dataset,and thefindings showed that it outperformed other recent approaches significantly.

Research on a Fog Computing Architecture and BP Algorithm Application for Medical Big Data

Although the Internet of Things has been widely applied,the problems of cloud computing in the application of digital smart medical Big Data collection,processing,analysis,and storage remain,especially the low efficiency of medical diagnosis.And with the wide application of the Internet of Things and Big Data in the medical field,medical Big Data is increasing in geometric magnitude resulting in cloud service overload,insufficient storage,communication delay,and network congestion.In order to solve these medical and network problems,a medical big-data-oriented fog computing architec-ture and BP algorithm application are proposed,and its structural advantages and characteristics are studied.This architecture enables the medical Big Data generated by medical edge devices and the existing data in the cloud service center to calculate,compare and analyze the fog node through the Internet of Things.The diagnosis results are designed to reduce the business processing delay and improve the diagnosis effect.Considering the weak computing of each edge device,the artificial intelligence BP neural network algorithm is used in the core computing model of the medical diagnosis system to improve the system computing power,enhance the medical intelligence-aided decision-making,and improve the clinical diagnosis and treatment efficiency.In the application process,combined with the characteristics of medical Big Data technology,through fog architecture design and Big Data technology integration,we could research the processing and analysis of heterogeneous data of the medical diagnosis system in the context of the Internet of Things.The results are promising:The medical platform network is smooth,the data storage space is sufficient,the data processing and analysis speed is fast,the diagnosis effect is remarkable,and it is a good assistant to doctors’treatment effect.It not only effectively solves the problem of low clinical diagnosis,treatment efficiency and quality,but also reduces the waiting time of patients,effectively solves the contradiction between doctors and patients,and improves the medical service quality and management level.

The Interdisciplinary Research of Big Data and Wireless Channel: A Cluster-Nuclei Based Channel Model

Recently,internet stimulates the explosive progress of knowledge discovery in big volume data resource,to dig the valuable and hidden rules by computing.Simultaneously,the wireless channel measurement data reveals big volume feature,considering the massive antennas,huge bandwidth and versatile application scenarios.This article firstly presents a comprehensive survey of channel measurement and modeling research for mobile communication,especially for 5th Generation(5G) and beyond.Considering the big data research progress,then a cluster-nuclei based model is proposed,which takes advantages of both the stochastical model and deterministic model.The novel model has low complexity with the limited number of cluster-nuclei while the cluster-nuclei has the physical mapping to real propagation objects.Combining the channel properties variation principles with antenna size,frequency,mobility and scenario dug from the channel data,the proposed model can be expanded in versatile application to support future mobile research.

Foundation Study on Wireless Big Data: Concept, Mining, Learning and Practices

Facing the development of future 5 G, the emerging technologies such as Internet of things, big data, cloud computing, and artificial intelligence is enhancing an explosive growth in data traffic. Radical changes in communication theory and implement technologies, the wireless communications and wireless networks have entered a new era. Among them, wireless big data(WBD) has tremendous value, and artificial intelligence(AI) gives unthinkable possibilities. However, in the big data development and artificial intelligence application groups, the lack of a sound theoretical foundation and mathematical methods is regarded as a real challenge that needs to be solved. From the basic problem of wireless communication, the interrelationship of demand, environment and ability, this paper intends to investigate the concept and data model of WBD, the wireless data mining, the wireless knowledge and wireless knowledge learning(WKL), and typical practices examples, to facilitate and open up more opportunities of WBD research and developments. Such research is beneficial for creating new theoretical foundation and emerging technologies of future wireless communications.

基于DBN深度学习算法的一站式诉求响应预测方法

为了提高诉求响应的速度,提出了基于机器学习的一站式诉求响应技术。在物理架构中采用事故数据记录器(ADR)服务器和数字化X线摄影术(DR)运行管理,实现一站式诉求响应;利用建模工具来构建例图进行描述诉求响应的运行细节,通过逻辑架构的感知层、网络层和应用层,实现了对一站式诉求响应的逻辑分析;利用机器学习预测方式和深度置信网络(DBN),实现一站式诉求响应的预测。实验表明,在进行对响应的速度进行测试时,所提出的系统响应所需时间最少为1.1 s,在进行对响应预测的准确性测试时,响应预测的准确性最高为97%。

A Survey of Machine Learning for Big Data Processing

Today’s world is a data-driven one,with data being produced in vast amounts as a result of the rapid growth of technology that permeates every aspect of our lives.New data processing techniques must be developed and refined over time to gain meaningful insights from this vast continuous volume of produced data in various forms.Machine learning technologies provide promising solutions and potential methods for processing large quantities of data and gaining value from it.This study conducts a literature review on the application of machine learning techniques in big data processing.It provides a general overview of machine learning algorithms and techniques,a brief introduction to big data,and a discussion of related works that have used machine learning techniques in a variety of sectors to process big amounts of data.The study also discusses the challenges and issues associated with the usage of machine learning for big data.

Modeling potential wetland distributions in China based on geographic big data and machine learning algorithms

Climate change and human activities have reduced the area and degraded the functions and services of wetlands in China.To protect and restore wetlands,it is urgent to predict the spatial distribution of potential wetlands.In this study,the distribution of potential wetlands in China was simulated by integrating the advantages of Google Earth Engine with geographic big data and machine learning algorithms.Based on a potential wetland database with 46,000 samples and an indicator system of 30 hydrologic,soil,vegetation,and topographic factors,a simulation model was constructed by machine learning algorithms.The accuracy of the random forest model for simulating the distribution of potential wetlands in China was good,with an area under the receiver operating characteristic curve value of 0.851.The area of potential wetlands was 332,702 km^(2),with 39.0%of potential wetlands in Northeast China.Geographic features were notable,and potential wetlands were mainly concentrated in areas with 400-600 mm precipitation,semi-hydric and hydric soils,meadow and marsh vegetation,altitude less than 700 m,and slope less than 3°.The results provide an important reference for wetland remote sensing mapping and a scientific basis for wetland management in China.

Deep Learning Based Optimal Multimodal Fusion Framework for Intrusion Detection Systems for Healthcare Data

Data fusion is a multidisciplinary research area that involves different domains.It is used to attain minimum detection error probability and maximum reliability with the help of data retrieved from multiple healthcare sources.The generation of huge quantity of data from medical devices resulted in the formation of big data during which data fusion techniques become essential.Securing medical data is a crucial issue of exponentially-pacing computing world and can be achieved by Intrusion Detection Systems(IDS).In this regard,since singularmodality is not adequate to attain high detection rate,there is a need exists to merge diverse techniques using decision-based multimodal fusion process.In this view,this research article presents a new multimodal fusion-based IDS to secure the healthcare data using Spark.The proposed model involves decision-based fusion model which has different processes such as initialization,pre-processing,Feature Selection(FS)and multimodal classification for effective detection of intrusions.In FS process,a chaotic Butterfly Optimization(BO)algorithmcalled CBOA is introduced.Though the classic BO algorithm offers effective exploration,it fails in achieving faster convergence.In order to overcome this,i.e.,to improve the convergence rate,this research work modifies the required parameters of BO algorithm using chaos theory.Finally,to detect intrusions,multimodal classifier is applied by incorporating three Deep Learning(DL)-based classification models.Besides,the concepts like Hadoop MapReduce and Spark were also utilized in this study to achieve faster computation of big data in parallel computation platform.To validate the outcome of the presented model,a series of experimentations was performed using the benchmark NSLKDDCup99 Dataset repository.The proposed model demonstrated its effective results on the applied dataset by offering the maximum accuracy of 99.21%,precision of 98.93%and detection rate of 99.59%.The results assured the betterment of the proposed model.

Energy Theft Detection in Smart Grids with Genetic Algorithm-Based Feature Selection

As big data,its technologies,and application continue to advance,the Smart Grid(SG)has become one of the most successful pervasive and fixed computing platforms that efficiently uses a data-driven approach and employs efficient information and communication technology(ICT)and cloud computing.As a result of the complicated architecture of cloud computing,the distinctive working of advanced metering infrastructures(AMI),and the use of sensitive data,it has become challenging tomake the SG secure.Faults of the SG are categorized into two main categories,Technical Losses(TLs)and Non-Technical Losses(NTLs).Hardware failure,communication issues,ohmic losses,and energy burnout during transmission and propagation of energy are TLs.NTL’s are human-induced errors for malicious purposes such as attacking sensitive data and electricity theft,along with tampering with AMI for bill reduction by fraudulent customers.This research proposes a data-driven methodology based on principles of computational intelligence as well as big data analysis to identify fraudulent customers based on their load profile.In our proposed methodology,a hybrid Genetic Algorithm and Support Vector Machine(GA-SVM)model has been used to extract the relevant subset of feature data from a large and unsupervised public smart grid project dataset in London,UK,for theft detection.A subset of 26 out of 71 features is obtained with a classification accuracy of 96.6%,compared to studies conducted on small and limited datasets.

A Hybrid Spatial Dependence Model Based on Radial Basis Function Neural Networks (RBFNN) and Random Forest (RF)

The majority of spatial data reveal some degree of spatial dependence. The term “spatial dependence” refers to the tendency for phenomena to be more similar when they occur close together than when they occur far apart in space. This property is ignored in machine learning (ML) for spatial domains of application. Most classical machine learning algorithms are generally inappropriate unless modified in some way to account for it. In this study, we proposed an approach that aimed to improve a ML model to detect the dependence without incorporating any spatial features in the learning process. To detect this dependence while also improving performance, a hybrid model was used based on two representative algorithms. In addition, cross-validation method was used to make the model stable. Furthermore, global moran’s I and local moran were used to capture the spatial dependence in the residuals. The results show that the HM has significant with a R2 of 99.91% performance compared to RBFNN and RF that have 74.22% and 82.26% as R2 respectively. With lower errors, the HM was able to achieve an average test error of 0.033% and a positive global moran’s of 0.12. We concluded that as the R2 value increases, the models become weaker in terms of capturing the dependence.

基于大数据流水线系统的算法模型整合方法研究——以基于机器学习方法的LiDAR数据树木生物量反演为例

【背景】激光雷达(LiDAR)数据在森林资源分析利用方面有着广泛应用,科研人员研制了很多涉及大数据管理和人工智能的专业算法模型,这些算法模型目前多数散落在研究人员手里,尚缺乏新型信息化平台对其进行整合。【方法】大数据流水线系统πFlow软件具有大数据管理能力和大数据算法集成能力,并可以所见即所得方式构建流水线并调度运行流水线,适合于LiDAR数据复杂算法模型的整合,且流水线可定制、可复用。【内容】本文介绍了πFlow的特点和功能,并以基于LiDAR冠层高度模型(CHM)数据的树冠解析及利用机器学习方法估测树木生物量为例,介绍了将算法整合到πFlow并构建LiDAR数据分析处理流水线的方法和技术,且对流水线进行了测试运行。【结果】利用πFlow构建的可重复信息化平台可支撑野外站观测网络的LiDAR数据生物量快速反演,为数据密集型的专业数据处理算法模型的整合提供了创新方法技术。

特色农产品销售评价大数据的弱监督分析方法

针对特色农产品评价大数据多维度分析中,可信标签不足以及挖掘消费者各维度真实情感语义困难等问题。该研究提出了一种基于弱监督训练的深度学习方法。首先,通过主题模型分析大规模评论,提取产品评价主题和关键词。然后,结合句法依存和情感词典为评论生成不同维度的伪标签。最后,构建多标签多分类深度网络,在伪标签上进行弱监督学习。结果表明,该方法在红心柚评论数据集上取得89.2%的准确率和80.3%的F1值,比随机森林算法提升了7.1个百分点的准确率和11.5个百分点的F1值。相比Transformer模型,准确率提高5.6个百分点,F1值提高2.0个百分点,参数量减少了92%。该方法能从海量评论中高效提取产品评价维度和消费者关注点,为完善农产品质量和销售服务提供数据支持。

空间自回归模型下不完整大数据缺失值插补算法

针对不完整大数据因其自身结构具有不规则性,导致在进行缺失值插补时计算量大、插补精度低的问题,提出空间自回归模型下不完整大数据缺失值插补算法。利用迁移学习算法在动态权重下过滤出原始数据中冗余数据,区分异常和正常数据,提取残缺数据,采用最小二乘回归对残缺数据实施修补。将缺失值插补分为3种类型,分别为一阶空间自回归模型插补、空间自回归模型插补和多重插补法。根据实际情况将修补后数据插补到合适的位置,实现不完整大数据缺失值插补。实验结果表明,所提方法具有良好的缺失值插补能力。

不确定大数据流分类的决策树模型构建仿真

在不确定大数据流分类过程中,受噪声和孤立点的干扰,导致处理效果和分类精度无法达到预期要求。为解决上述问题,提出一种基于决策树模型的不确定大数据流分类算法。通过采用在线字典学习算法,对不确定大数据流去噪处理,消除噪声对分类过程产生的干扰。构建决策树,在剪枝过程中通过特征过滤算法,滤除不确定大数据流中掺杂的孤立点。将去噪后的不确定大数据流,输入决策树模型中,完成分类工作。实验结果表明,所提算法处理后的不确定大数据流振幅明显减小,且分类精度高,具有一定的应用价值。

颗粒度动态控制的负载均衡算法的大数据分析

本文提出了一种创新的动态颗粒度控制算法,此算法融合了延迟进程生成与负载内联的特点,通过构建数学模型、优化算法,解决了传统负载均衡方法在动态环境下处理并行任务的难题。旨在最大化并行处理效率并削减不必要的开销。为大数据分析和并行处理提供了一种高效、灵活的负载均衡方案,对提升系统性能与降低成本有一定的参考价值。

应用导向下的“统计学习方法”教学探索与实践

“统计学习方法”是以数据为研究对象、以计算机及网络为平台、通过统计理论建立模型并对数据进行分析和预测的学科,该课程的学习有助于提升学生的实践意识和应用意识,满足大数据时代下对统计数据分析人才的需求。本文在大数据环境下,以应用人才培养为导向,以统计数据分析方向的学生为服务对象,深入挖掘“统计学习方法”在专业课程体系中的地位和作用,构建了适应于数据分析背景的课堂教学和实验教学体系,有助于提升学生的创新思维和实践能力,培养多元化创新型人才。

基于联邦学习的政务大数据平台应用研究

当前数字政府建设已进入深水区,政务大数据平台作为数据底座支撑各类政务信息化应用,其隐私数据的安全性和合规性一直被业界广泛关注。联邦学习是一类解决数据孤岛的重要方法,基于联邦学习的政务一体化大数据平台应用具有较高的研究价值。首先,介绍政务大数据平台及联邦学习应用现状;然后,分析政务大数据平台面临的隐私数据的采集、分类分级、共享三大管理挑战;接着,阐述基于联邦学习的推荐算法和隐私集合求交技术的解决方法;最后,对政务大数据平台隐私数据的未来应用进行了总结和展望。

基于挖掘算法的用户大数据周期智能推荐仿真

随着互联网技术的快速发展,用户数量日益增长,社交网络平台对周期性智能推荐的需求也日益增加。为了解决当前智能推荐算法准确率低、推荐速度慢等问题,提出了一种基于挖掘算法的用户大数据周期智能推荐算法。算法首先采用协同推荐算法对用户历史行为进行分析,并通过数据相似性衡量智能推荐的效果,使用Top-N算法优化推荐过程,达到周期智能推荐的目的;然后采用基于神经网络的挖掘算法对智能推荐算法进行优化,挖掘长时数据关系的同时保持短时数据之间的非线性;最后通过引入灰色均衡算法对相似度计算优化,从而缩短推荐时间。实验结果表明,所提算法在相似度计算准确度方面提升7%,推荐精确度提升6%,召回率提升8%,有效地提高了数据周期智能推荐的精读和效率,提高了个性化服务的质量。

基于CSA-PLS算法的养殖水体水质快速高光谱预测反演模型研究

养殖水体水质的优劣直接影响养殖对象的成长,准确、快速、全面地掌控养殖水环境的水质参数变化情况具有重要意义。传统的水质指标监测方法都通过人工采样的方式,不仅耗费时间长,且只能体现局部水体情况。针对这些问题,提出了一种乌鸦搜索算法(CSA)结合偏最小二乘回归(PLSR)的高光谱特征波段筛选方法,快速构建回归模型,实现光谱数据的精准预测反演。以连片的养殖小区为研究对象,采集养殖水体样本并拍摄同时期的高光谱影像数据。首先对提取的采样点光谱数据利用多种数据变换方法分别预处理;其次利用这些数据,对水质指标总氮(TN)、氨氮(NH_(4)^(+)-N)、总磷(TP)和化学需氧量(COD)分别构建全波段的SVR和AdaBoost回归模型,同时与提出的CSA-PLS自动筛选波段方法和传统的连续投影算法(SPA)筛选波段后构建的模型进行比较分析;最后根据决定系数(R^(2))和均方根误差(REMS)选出适合各水质指标的最优模型。从实验结果可以看出,所提波段筛选方法的AdaBoost模型预测结果优于SVR和传统SPA方法提取特征波段后构建的模型,与全波段最优模型相比,在评价指标R^(2)和RMSE上TN提升了18.32%和10.73%;NH_(4)^(+)-N提升了17.42%和11.19%;COD提升了2.15%和2.54%。结果表明,基于CSA-PLS算法的光谱波段自动筛选方法结合AdaBoost构建的预测反演模型是有效、可行的,具有较高的精准度,为实现养殖水环境实时准确的预警调控提供了一种新的数据预测模型。

一种采用渐进学习模式的SBS-CLearning分类算法

针对Learn++. NSE算法中多个基分类器之间相互独立、未利用前阶段学习结果辅助后续阶段学习而准确率较低的问题,借鉴人类的学习过程,优化Learn++. NSE算法内部的学习机制,转变基分类器的独立学习为渐进学习,提出了一种采用渐进学习模式的SBS-CLearning分类算法.分析了Learn++. NSE算法的不足.给出了SBS-CLearning算法的步骤,该算法在前阶段基分类器的基础之上先增量学习,再完成最终的加权集成.在测试数据集上对比分析了Learn++. NSE与SBSCLearning的分类准确率.试验结果表明:SBS-CLearning算法吸收了增量学习与集成学习的优势,相比Learn++. NSE提高了分类准确率.针对SEA人工数据集,SBS-CLearning,Learn++. NSE的平均分类准确率分别为0. 982,0. 976.针对旋转棋盘真实数据集,在Constant,Sinusoidal,Pulse环境下,SBS-CLearning的平均分类准确率分别为0. 624,0. 655,0. 662,而Learn++. NSE分别为0. 593,0. 633,0. 629.