期刊文献+
共找到7,491篇文章
< 1 2 250 >
每页显示 20 50 100
ANALYSIS OF A TORNADO-LIKE SEVERE STORM IN THE OUTER REGION OF THE 2007 SUPER TYPHOON SEPAT 被引量:3
1
作者 郑峰 陈联寿 钟建锋 《Journal of Tropical Meteorology》 SCIE 2011年第2期175-180,共6页
When super typhoon Sepat came close to the Fujian coastline on the night of 18 August 2007 (coded as 0709 in Chinese convention), an associated tornado-like severe storm developed at 2307–2320 Beijing Standard Time i... When super typhoon Sepat came close to the Fujian coastline on the night of 18 August 2007 (coded as 0709 in Chinese convention), an associated tornado-like severe storm developed at 2307–2320 Beijing Standard Time in Longgang, Cangnan County, Wenzhou Prefecture, Zhejiang Province approximately 300 km away in the forward direction of the typhoon. The storm caused heavy losses in lives and property. Studying the background of the formation of the storm, this paper identifies some of its typical characteristics after analyzing its retrieval of Doppler radar data, vertical wind shear and so on. Synoptic conditions, such as unstable weather processes and TBB, are also studied. 展开更多
关键词 tornado-like severe storms radar characteristics OBSERVATION
下载PDF
Storm分布式计算框架下基于知识图谱的快速学习资源推荐 被引量:2
2
作者 刘莹 杨淑萍 张治国 《南京邮电大学学报(自然科学版)》 北大核心 2024年第3期93-99,共7页
针对在线学习资源推荐存在精度较低或实时性较差的问题,采用知识图谱进行用户及资源的知识表示,并采用长短时间记忆网络对用户资源特征差进行优化,从而将与用户特征差最小的资源推送给用户。首先,在获得在线学习记录样本后,利用知识图... 针对在线学习资源推荐存在精度较低或实时性较差的问题,采用知识图谱进行用户及资源的知识表示,并采用长短时间记忆网络对用户资源特征差进行优化,从而将与用户特征差最小的资源推送给用户。首先,在获得在线学习记录样本后,利用知识图谱进行实体特征关系的知识表示,并借助Storm分布式框架生成知识图谱中头尾实体及关系特征向量。接着,建立用户-资源实体的最小特征差目标函数,并采用长短时间记忆网络对最小特征差目标函数进行优化。最后,通过Storm分布式平台进行长短时间记忆网络的参数求解,从而快速生成稳定的相关资源推荐模型。实验结果表明,在Storm分布式框架下采用知识图谱和长短时间记忆网络实现在线资源推荐,可获得较高准确率及运行效率,在应对大规模资源的实时推荐方面具有较强的适应度。 展开更多
关键词 资源推荐 知识图谱 storm框架 长短时间记忆 TransD模型
下载PDF
Machine Learning Analysis of Impact of Western US Fires on Central US Hailstorms 被引量:1
3
作者 Xinming LIN Jiwen FAN +1 位作者 Yuwei ZHANG ZJason HOU 《Advances in Atmospheric Sciences》 SCIE CAS CSCD 2024年第7期1450-1462,共13页
Fires,including wildfires,harm air quality and essential public services like transportation,communication,and utilities.These fires can also influence atmospheric conditions,including temperature and aerosols,potenti... Fires,including wildfires,harm air quality and essential public services like transportation,communication,and utilities.These fires can also influence atmospheric conditions,including temperature and aerosols,potentially affecting severe convective storms.Here,we investigate the remote impacts of fires in the western United States(WUS)on the occurrence of large hail(size:≥2.54 cm)in the central US(CUS)over the 20-year period of 2001–20 using the machine learning(ML),Random Forest(RF),and Extreme Gradient Boosting(XGB)methods.The developed RF and XGB models demonstrate high accuracy(>90%)and F1 scores of up to 0.78 in predicting large hail occurrences when WUS fires and CUS hailstorms coincide,particularly in four states(Wyoming,South Dakota,Nebraska,and Kansas).The key contributing variables identified from both ML models include the meteorological variables in the fire region(temperature and moisture),the westerly wind over the plume transport path,and the fire features(i.e.,the maximum fire power and burned area).The results confirm a linkage between WUS fires and severe weather in the CUS,corroborating the findings of our previous modeling study conducted on case simulations with a detailed physics model. 展开更多
关键词 WILDFIRE severe convective storm HAILstorm machine learning
下载PDF
Tornado-like non-stationary vortices: experimental modelling under laboratory conditions
4
作者 Viktor N. Kopeitsev Mikhael E. Romash Aleksei Y. Varaksin 《Natural Science》 2011年第11期907-913,共7页
The purposes of the paper are threefold: 1) to show the fundamental possibility of physical simulation of non-stationary wall-free concentrated air vortices (called here as tornado-like vortices) under laboratory cond... The purposes of the paper are threefold: 1) to show the fundamental possibility of physical simulation of non-stationary wall-free concentrated air vortices (called here as tornado-like vortices) under laboratory conditions without using of the mechanical rotation, 2) to find the heating rates of underlying surface and air temporal and spatial temperature shear which lead to the stable genesis of wall-free vortices, and 3) to investigate the some parameters of the process of generation of vortices and their characteristics. The wall-free vortices were generated over underlying surface of aluminum sheet due to its controlled heating from below as a result of development of unstable stratifycation of air. 展开更多
关键词 VORTEX FLOWS tornado-like FLOWS Physical Simulation
下载PDF
Thermosphere joint observations by TM-1 constellations and Swarm-B during the April 2023 geomagnetic storm
5
作者 YongPing Li YueQiang Sun +9 位作者 XianGuo Zhang JiangZhao Ai XiaoLiang Zheng Jia Li YuJie Wang BiBo Guo Feng Yan ShiLong Wei XinChun Tang YuanYuan Cao 《Earth and Planetary Physics》 EI CAS CSCD 2024年第2期307-316,共10页
The response of thermosphere density to geomagnetic storms is a complicated physical process.Multi-satellite joint observations at the same altitude but different local times(LTs)are important for understanding this p... The response of thermosphere density to geomagnetic storms is a complicated physical process.Multi-satellite joint observations at the same altitude but different local times(LTs)are important for understanding this process;however,until now such studies have hardly been done.In this report,we analyze in detail the thermosphere mass density response at 510 km during the April 23−24,2023 geomagnetic storm using data derived from the TM-1(TianMu-1)satellite constellation and Swarm-B satellites.The observations show that there were significant LT differences in the hemispheric asymmetry of the thermosphere mass density during the geomagnetic storm.Densities observed by satellite TM02 at nearly 11.3 and 23.3 LTs were larger in the northern hemisphere than in the southern.The TM04 dayside density observations appear to be almost symmetrical with respect to the equator,though southern hemisphere densities on the nightside were higher.Swarm-B data exhibit near-symmetry between the hemispheres.In addition,the mass density ratio results show that TM04 nightside observations,TM02 data,and Swarm-B data all clearly show stronger effects in the southern hemisphere,except for TM04 on the dayside,which suggest hemispheric near-symmetry.The South-North density enhancement differences in TM02 and TM04 on dayside can reach 130%,and Swarm-B data even achieve 180%difference.From the observations of all three satellites,large-scale traveling atmospheric disturbances(TADs)first appear at high latitudes and propagate to low latitudes,thereby disturbing the atmosphere above the equator and even into the opposite hemisphere.NRLMSISE00 model simulations were also performed on this geomagnetic storm.TADs are absent in the NRLMSISE00 simulations.The satellite data suggest that NRLMSISE00 significantly underestimates the magnitude of the density response of the thermosphere during geomagnetic storms,especially at high latitudes in both hemispheres.Therefore,use of the density simulation of NRLMSISE00 may lead to large errors in satellite drag calculations and orbit predictions.We suggest that the high temporal and spatial resolution of direct density observations by the TM-1 constellation satellites can provide an autonomous and reliable basis for correction and improvement of atmospheric models. 展开更多
关键词 TM-1 constellation Swarm-B joint observations geomagnetic storm Local Times
下载PDF
Trend of Storm Surge Induced by Typical Landfall Super Typhoons During 1975–2021 in the Eastern China Sea
6
作者 LUO Feng WANG Yi +3 位作者 TAO Aifeng SHI Jian WANG Yongzhi ZHANG Chi 《Journal of Ocean University of China》 CAS CSCD 2024年第2期277-286,共10页
Climate change affects the activity of global and regional tropical cyclones(TCs).Among all TCs,typical super typhoons(STYs)are particularly devastating because they maintain their intensity when landing on the coast ... Climate change affects the activity of global and regional tropical cyclones(TCs).Among all TCs,typical super typhoons(STYs)are particularly devastating because they maintain their intensity when landing on the coast and thus cause casualties,economic losses,and environmental damage.Using a 3D tidal model,we reconstructed the typhoon(TY)wind field to simulate the storm surge induced by typical STYs.The TY activity was then analyzed using historical data.Results showed a downtrend of varying degrees in the annual frequency of STYs and TCs in the Western North Pacific(WNP)Basin,with a significant trend change observed for TCs from 1949 to 2021.A large difference in the interannual change in frequency was found between STYs and TCs in the WNP and Eastern China Sea(ECS).Along the coast of EC,the frequency of landfall TCs showed a weak downtrend,and the typical STYs showed reverse micro growth with peak activity in August.Zhejiang,Fujian,and Taiwan were highly vulnerable to the frontal hits of typical STYs.Affected by climate change,the average lifetime maximum intensity(LMI)locations and landfall locations of typical STYs in the ECS basin showed a significant poleward migration trend.In addition,the annual average LMI and accumulated cyclone energy showed an uptrend,indicating the increasing severity of the disaster risk.Affected by the typical STY activity in the ECS,the maximum storm surge area also showed poleward migration,and the coast of North China faced potential growth in high storm surge risks. 展开更多
关键词 storm surge super typhoons tropical cyclones eastern China Sea poleward migration
下载PDF
Recent geomagnetic storms observed by Macao Science Satellite-1
7
作者 Qing Yan HongBo Yao 《Earth and Planetary Physics》 EI CAS CSCD 2024年第4期565-569,共5页
Geomagnetic storms are rapid disturbances of the Earth’s magnetosphere.They are related to many geophysical phenomena and have large influences on human activities.Observing and studying geomagnetic storms is thus of... Geomagnetic storms are rapid disturbances of the Earth’s magnetosphere.They are related to many geophysical phenomena and have large influences on human activities.Observing and studying geomagnetic storms is thus of great significance to both scientific research and geomagnetic hazards prevention.The Macao Science Satellite-1(MSS-1)project includes two high-precision Chinese geomagnetic satellites successfully launched on May 21,2023.The main purpose of MSS-1 is to accurately measure the Earth’s magnetic field.Here,we analyze early MSS-1 geomagnetic field measurements and report observations of two recent geomagnetic storms that occurred on March 24,2024 and May 11,2024.We also calculate the related geoelectric fields as an initial step towards a quantitative assessment of geomagnetic hazards. 展开更多
关键词 Macao Science Satellite-1 geomagnetic storms satellite magnetics electromagnetic induction geoelectric fields
下载PDF
A typhoon-induced storm surge numerical model with GPU acceleration based on an unstructured spherical centroidal Voronoi tessellation grid
8
作者 Yuanyong Gao Fujiang Yu +2 位作者 Cifu Fu Jianxi Dong Qiuxing Liu 《Acta Oceanologica Sinica》 SCIE CAS CSCD 2024年第3期40-47,共8页
Storm surge is often the marine disaster that poses the greatest threat to life and property in coastal areas.Accurate and timely issuance of storm surge warnings to take appropriate countermeasures is an important me... Storm surge is often the marine disaster that poses the greatest threat to life and property in coastal areas.Accurate and timely issuance of storm surge warnings to take appropriate countermeasures is an important means to reduce storm surge-related losses.Storm surge numerical models are important for storm surge forecasting.To further improve the performance of the storm surge forecast models,we developed a numerical storm surge forecast model based on an unstructured spherical centroidal Voronoi tessellation(SCVT)grid.The model is based on shallow water equations in vector-invariant form,and is discretized by Arakawa C grid.The SCVT grid can not only better describe the coastline information but also avoid rigid transitions,and it has a better global consistency by generating high-resolution grids in the key areas through transition refinement.In addition,the simulation speed of the model is accelerated by using the openACC-based GPU acceleration technology to meet the timeliness requirements of operational ensemble forecast.It only takes 37 s to simulate a day in the coastal waters of China.The newly developed storm surge model was applied to simulate typhoon-induced storm surges in the coastal waters of China.The hindcast experiments on the selected representative typhoon-induced storm surge processes indicate that the model can reasonably simulate the distribution characteristics of storm surges.The simulated maximum storm surges and their occurrence times are consistent with the observed data at the representative tide gauge stations,and the mean absolute errors are 3.5 cm and 0.6 h respectively,showing high accuracy and application prospects. 展开更多
关键词 typhoon-induced storm surge numerical model GPU acceleration unstructured grid spherical centroidal Voronoi tessellation(SCVT)
下载PDF
The dawn−dusk asymmetry in mesosphere and lower thermosphere temperature disturbances during geomagnetic storms at high latitude
9
作者 GuanChun Wei JianYong Lu +2 位作者 Fen Tang JingYuan Li Meng Sun 《Earth and Planetary Physics》 EI CAS CSCD 2024年第2期356-367,共12页
Utilizing observations by the Sounding of the Atmosphere using Broadband Emission Radiometry(SABER)instrument,we quantitatively assessed the dawn-dusk asymmetry in temperature disturbances within the high-latitude mes... Utilizing observations by the Sounding of the Atmosphere using Broadband Emission Radiometry(SABER)instrument,we quantitatively assessed the dawn-dusk asymmetry in temperature disturbances within the high-latitude mesosphere and lower thermosphere(MLT)during the main phase of geomagnetic storms in this study.An analysis of five geomagnetic superstorm events indicated that during the main phase,negative temperature disturbances were more prevalent on the dawn side than on the dusk side in the high-latitude MLT region.Results of a statistical analysis of 54 geomagnetic storm events also revealed a notable disparity in temperature disturbances between the dawn and dusk sides.At high latitudes,38.2%of the observational points on the dawn side exhibited negative temperature disturbances(less than−5 K),whereas on the dusk side,this percentage was only 29.5%.In contrast,at mid-latitudes,these proportions were 34.1%and 36.5%,respectively,showing no significant difference.We also conducted a statistical analysis of temperature disturbances at different altitudes,which revealed an increase in the proportion of warming disturbances with altitude.Conversely,the proportion of cooling disturbances initially rose with altitude,reaching a peak around 105 km,and subsequently decreased.These temperature disturbance differences could be explained by the day-night asymmetry in vertical wind disturbances during storm conditions. 展开更多
关键词 DAWN dusk asymmetry SABER(Sounding of the Atmosphere using Broadband Emission Radiometry) geomagnetic storms mesosphere and lower thermosphere
下载PDF
基于Storm云平台的电力设备并行故障诊断方法
10
作者 刘少伟 《电工技术》 2024年第14期132-135,共4页
为了实现电力行业多源数据的监测诊断,应对电力系统的实时处理需求,引入Storm分布式实时计算平台对数据进行处理。在此平台上部署基于混合聚类的流数据处理模型,实现数据流的故障诊断。通过将减法聚类算法和K-means算法结合实现流式数... 为了实现电力行业多源数据的监测诊断,应对电力系统的实时处理需求,引入Storm分布式实时计算平台对数据进行处理。在此平台上部署基于混合聚类的流数据处理模型,实现数据流的故障诊断。通过将减法聚类算法和K-means算法结合实现流式数据的故障检测。减法聚类获取较优的聚类中心,而K-means算法根据此聚类中心计算出较好的分类结果。测试集群和单机的数据处理量的结果表明在集群环境下合理设置组件并行度可以提高流计算时效性和吞吐量。 展开更多
关键词 故障诊断 storm云平台 减法聚类 K-MEANS
下载PDF
Risk assessment of coastal flooding disaster by storm surge based on Elevation-Area method and hydrodynamic models:Taking Bohai Bay as an example
11
作者 Fu Wang Xue-zheng Liu +3 位作者 Yong Li Heng Yu Ming-zheng Wen Yun-zhuang Hu 《China Geology》 CAS CSCD 2024年第3期494-504,共11页
The future inundation by storm surge on coastal areas are currently ill-defined.With increasing global sealevel due to climate change,the coastal flooding by storm surge is more and more frequently,especially in coast... The future inundation by storm surge on coastal areas are currently ill-defined.With increasing global sealevel due to climate change,the coastal flooding by storm surge is more and more frequently,especially in coastal lowland with land subsidence.Therefore,the risk assessment of such inundation for these areas is of great significance for the sustainable socio-economic development.In this paper,the authors use Elevation-Area method and Regional Ocean Model System(ROMS)model to assess the risk of the inundation of Bohai Bay by storm surge.The simulation results of Elevation-Area method show that either a 50-year or 100-year storm surge can inundate coastal areas exceeding 8000 km^(2);the numerical simulation results based on hydrodynamics,considering ground friction and duration of the storm surge high water,show that a 50-year or 100-year storm surge can only inundate an area of over 2000 km^(2),which is far less than 8000 km^(2);while,when taking into account the land subsidence and sea level rise,the very inundation range will rapidly increase by 2050 and 2100.The storm surge will greatly impact the coastal area within about 10-30 km of the Bohai Bay,in where almost all major coastal projects are located.The prompt response to flood disaster due to storm surge is urgently needed,for which five suggestions have been proposed based on the geological background of Bohai Bay.This study may offer insight into the development of the response and adaptive plans for flooding disasters caused by storm surge. 展开更多
关键词 storm surge ROMS Elevation-Area method Numerical simulation Land subsidence Flooding disaster Sea level rise Marine geological survey engineering Geological disaster survey engineering Bohai Bay
下载PDF
Anta Storm Mecha Jacket:Pioneering domestic waterproof tech
12
《China Textile》 2024年第2期13-13,共1页
The Anta Storm Mecha Jacket is a rainproof and comfortable jacket launched by Anta and Donghua University.It is equipped with China's first self-developed high-performance waterproof and breathable material-Aerove... The Anta Storm Mecha Jacket is a rainproof and comfortable jacket launched by Anta and Donghua University.It is equipped with China's first self-developed high-performance waterproof and breathable material-Aerovent.This jacket not only has excellent waterproof performance,but also achieves a soft,breathable and comfortable wearing experience.After testing by an authoritative third-party organization,its waterproof and breathable performance is comparable to international brands. 展开更多
关键词 storm PIONEER has
下载PDF
基于Storm的可扩展的分布式网络爬虫系统设计研究
13
作者 池国俊 《电脑知识与技术》 2024年第20期89-91,共3页
为提高分布式网络爬虫的速度和性能,文章应用Storm云平台,提出了分布式网络爬虫系统的设计方案。首先,针对传统单机网络爬虫技术过于落后,无法满足现代化日益增长的数据抓取使用需求等问题,严格遵循系统设计原则,运用Storm云平台等技术... 为提高分布式网络爬虫的速度和性能,文章应用Storm云平台,提出了分布式网络爬虫系统的设计方案。首先,针对传统单机网络爬虫技术过于落后,无法满足现代化日益增长的数据抓取使用需求等问题,严格遵循系统设计原则,运用Storm云平台等技术,完成了系统框架设计、系统数据库以及种子URL文档、模拟登录、URL优化链接、网页下载等模块的设计,确保系统功能实现效果。最后,对系统设计方案的执行效果进行了测试。结果表明:文章提出的系统设计方案具有较高的可靠性和可行性,各个功能模块运行良好,其准确率达到百分之百,使得本系统具有可靠性高、稳定性强、数据处理效率高等特点,完全符合预期的设计标准和要求。 展开更多
关键词 storm 可扩展 分布式 网络爬虫 系统设计
下载PDF
大数据流式计算框架Storm的任务迁移策略 被引量:25
14
作者 鲁亮 于炯 +3 位作者 卞琛 刘月超 廖彬 李慧娟 《计算机研究与发展》 EI CSCD 北大核心 2018年第1期71-92,共22页
Storm作为流式计算模式下最具代表性的平台之一,其默认轮询的调度机制未考虑到异构环境下不同工作节点的自身性能和负载差异,以及工作节点之间的网络传输开销和节点内部的进程与线程通信开销,无法充分发挥集群的性能.为了在各类资源约... Storm作为流式计算模式下最具代表性的平台之一,其默认轮询的调度机制未考虑到异构环境下不同工作节点的自身性能和负载差异,以及工作节点之间的网络传输开销和节点内部的进程与线程通信开销,无法充分发挥集群的性能.为了在各类资源约束的前提下最小化通信开销,在建立并论证Storm资源约束模型、最优通信开销模型和任务迁移模型的基础上,提出一种异构Storm环境下的任务迁移策略(task migration strategy for heterogeneous Storm cluster,TMSH-Storm),包括源节点选择算法和任务迁移算法.其中,源节点选择算法根据集群中各工作节点CPU、内存和网络带宽的负载情况以及各类资源的优先级顺序,将超出阈值的节点加入源节点集;任务迁移算法综合迁移开销、通信开销、节点资源约束以及节点和任务负载等因素,依次将源节点中的待迁移任务异步迁移至目的节点上.实验表明:相对于现有研究而言,TMSH-Storm能有效降低延迟和节点间通信开销,且执行开销较小. 展开更多
关键词 大数据 流式计算 storm 通信开销 任务迁移
下载PDF
Storm环境下基于权重的任务调度算法 被引量:15
15
作者 鲁亮 于炯 +3 位作者 卞琛 英昌甜 师康利 蒲勇霖 《计算机应用》 CSCD 北大核心 2018年第3期699-706,共8页
大数据流式计算平台Apache Storm默认采用轮询的方式进行任务调度,未考虑到拓扑中各任务计算开销的差异以及任务之间不同类型的通信模式,在负载均衡和通信开销方面存在较大的优化空间。针对这一问题,提出一种Storm环境下基于权重的任务... 大数据流式计算平台Apache Storm默认采用轮询的方式进行任务调度,未考虑到拓扑中各任务计算开销的差异以及任务之间不同类型的通信模式,在负载均衡和通信开销方面存在较大的优化空间。针对这一问题,提出一种Storm环境下基于权重的任务调度算法(TSAW-Storm)。该算法首先根据各任务的CPU资源占用情况以及任务间的数据流大小,分别确定拓扑的点权和边权;并利用最大化边权增益的思想,逐步构建起各工作节点中承载的任务集合,在保证集群负载均衡的同时,尽可能将边权较大的节点间数据流转化为节点内数据流,从而降低网络传输开销。实验结果表明,在包含有8个工作节点的WordCount基准测试中,TSAW-Storm的系统延迟和节点间数据流大小相比Storm默认调度算法分别降低了30.0%和32.9%,且各工作节点的CPU负载标准差仅为Storm默认调度算法的25.8%;此外,在与在线调度算法的对比实验中,TSAW-Storm在系统延迟、节点间数据流大小和CPU负载标准差方面分别降低了7.76%、11.8%和5.93%,且算法的执行开销明显降低,有效提高了Storm系统的运行效率。 展开更多
关键词 大数据 流式计算 storm 权重 任务调度 负栽均衡 通信开销
下载PDF
基于Storm的智能电网广域测量系统数据实时加密 被引量:15
16
作者 张少敏 孙婕 王保义 《电力系统自动化》 EI CSCD 北大核心 2016年第21期123-127,167,共6页
针对智能电网广域测量系统(WAMS)数据的安全问题,分析了现有数据加密方案的特点。结合WAMS数据量大、实时性要求高的特点,引入Storm分布式实时计算平台进行WAMS数据的实时加密处理。设计了基于云平台的WAMS数据存储系统,并在此系统之上... 针对智能电网广域测量系统(WAMS)数据的安全问题,分析了现有数据加密方案的特点。结合WAMS数据量大、实时性要求高的特点,引入Storm分布式实时计算平台进行WAMS数据的实时加密处理。设计了基于云平台的WAMS数据存储系统,并在此系统之上设计了基于Storm框架的高级加密标准(AES)算法。该算法将加密过程分为数据接入、快速并行数据加密及加密结果云存储等几个过程,并在Storm预定义的编程组件中进行了编程实现。通过对比分析,验证了所提出方案具有低延迟、加密效率高等特点。 展开更多
关键词 广域测量系统 云计算 storm 数据加密
下载PDF
基于Storm的在线序列极限学习机的气象预测模型 被引量:9
17
作者 欧阳建权 周勇 唐欢容 《计算机研究与发展》 EI CSCD 北大核心 2017年第8期1736-1743,共8页
为提高气象预测精度,实时应对频发的局域气象灾害,拥有更高的处理海量数据的效率,提出了一种基于Storm的在线序列的极限学习机气象预测模型.该模型首先初始化多个在线极限学习机,当新批次的数据不断到达时,模型能够在训练结果的基础上... 为提高气象预测精度,实时应对频发的局域气象灾害,拥有更高的处理海量数据的效率,提出了一种基于Storm的在线序列的极限学习机气象预测模型.该模型首先初始化多个在线极限学习机,当新批次的数据不断到达时,模型能够在训练结果的基础上继续学习新样本,并引入随机梯度下降法和误差权值调整方法,对新的预测结果进行误差反馈,实时更新误差权值参数,以提高模型预测准确率.另外,采用Storm流式处理框架对提出的算法模型进行并行化改进,以提高处理海量高维数据的能力.实验结果表明:该模型与基于Hadoop的并行极限学习机算法(parallel extreme learning machine,PELM)相比,具有更高的预测精度和优异的并行性能. 展开更多
关键词 storm 极限学习机 气象预测 在线序列 机器学习
下载PDF
Storm环境下一种改进的任务调度策略 被引量:8
18
作者 刘月超 于炯 鲁亮 《新疆大学学报(自然科学版)》 CAS 北大核心 2017年第1期90-95,共6页
随着大数据业务的快速增长,针对大规模数据处理的实时计算变成了一种业务上的需求,缺少"实时的Hadoop系统"经成为整个大数据生态系统中的一个巨大缺失,Storm的出现很好地满足了这一需求.然而,Storm默认的round-robin资源调度... 随着大数据业务的快速增长,针对大规模数据处理的实时计算变成了一种业务上的需求,缺少"实时的Hadoop系统"经成为整个大数据生态系统中的一个巨大缺失,Storm的出现很好地满足了这一需求.然而,Storm默认的round-robin资源调度,既不考虑内部组件的传输延迟时间,也不考虑节点间的传输延迟.因此,本文提出了Storm环境下一种改进的任务调度策略(Improved Storm Task Schedule Strategy,ISTS),该策略旨在提高通过最大限度地提高资源利用率和吞吐量同时最小化网络延迟.实验结果表明,使用ISTS进行任务调度时,可以同时满足软、硬资源限制以及最小化组件之间的网络传输距离. 展开更多
关键词 大数据 storm 数据处理 调度
下载PDF
基于Hadoop与Storm的日志实时处理系统研究 被引量:14
19
作者 李洋 吕家恪 《西南师范大学学报(自然科学版)》 CAS 北大核心 2017年第4期119-126,共8页
日志数据记录着丰富的信息,具有较高的实用价值,但在当今大数据时代环境下,数据量的陡增为日志数据的处理带来了挑战.为了有效地解决海量日志数据处理面临的瓶颈问题,本文整合Hadoop和Storm分布式框架,构建一种融合了实时计算与离线计... 日志数据记录着丰富的信息,具有较高的实用价值,但在当今大数据时代环境下,数据量的陡增为日志数据的处理带来了挑战.为了有效地解决海量日志数据处理面临的瓶颈问题,本文整合Hadoop和Storm分布式框架,构建一种融合了实时计算与离线计算的分布式日志实时处理系统.系统架构由数据服务层、业务逻辑层和Web展示层组成,数据服务层使用Flume实时采集日志数据,并分别采用Kafka与HBase完成实时日志流数据的缓冲和系统数据的持久化存储;业务逻辑层利用Storm对实时日志流数据进行实时分析,并使用Hadoop的计算引擎MapReduce结合数据挖掘技术完成对海量历史日志数据的离线分析,离线分析的结果为实时分析提供支持、参考;Web展示层负责日志数据及其分析结果的展示.实验结果表明,系统能有效地解决日志数据的采集存储、实时日志流数据的实时分析和历史日志数据的离线分析等问题,并成功地融合了Hadoop与Storm各自的优势,为日志数据的采集和分析系统的构建提供新的技术参考. 展开更多
关键词 日志 HADOOP FLUME storm HBASE
下载PDF
Storm启发式均衡图划分调度优化方法 被引量:5
20
作者 简琤峰 卢涛 张美玉 《小型微型计算机系统》 CSCD 北大核心 2018年第11期2538-2544,共7页
针对现有Storm调度器未考虑通信代价,负载不均、无法动态重调度等缺陷,提出了基于启发式均衡图划分算法的调度策略.通过对Storm建立调度模型,将负载检测作为调度器的输入实现动态并行参数优化和重调度优化,最终减少集群节点间的数据发送... 针对现有Storm调度器未考虑通信代价,负载不均、无法动态重调度等缺陷,提出了基于启发式均衡图划分算法的调度策略.通过对Storm建立调度模型,将负载检测作为调度器的输入实现动态并行参数优化和重调度优化,最终减少集群节点间的数据发送率,并且保持节点间负载均衡.实验表明本文算法建立的动态调度器能够有效减少数据处理延时,提升集群吞吐量,实现对集群性能的整体优化. 展开更多
关键词 storm 动态调度 图均衡划分 重调度
下载PDF
上一页 1 2 250 下一页 到第
使用帮助 返回顶部