Flow Structure and Short-Term Riverbed Evolution in Curved Flumes

River bending is the major effect responsible for bed topography and bank changes.In this study,fluid velocity(measured by a three-dimensional Doppler advanced point current meter)and bed topographical data have been collected in 40 sections of an experimental model.The whole flume was composed of an organic glass bend,upstream and downstream water tanks,two transition straight sections,a circulation pump,and a connection pipeline.Each section has been found to be characterized by a primary circulation and a small reverse circulation,with some sections even presenting three more or more circulation structures.The minimum circulation intensity has been detected in proximity to the top of the curved channel,while a region with small longitudinal velocity has been observed near the concave bank of each bend,corresponding to the flat bed formed after a short period of scouring.The maximum sediment deposition and scour depth in the presence of a uniform distribution of living flexible vegetation within 10 cm of the flume wall have been found to be smaller than those observed in the tests conducted without vegetation.

Effect of particle composition and consolidation degree on the wave-induced liquefaction of soil beds

The wave-induced liquefaction of seabed is responsible for causing damage to marine structures.Particle composition and consolidation degree are the key factors affecting the pore water pressure response and liquefaction behavior of the seabed under wave action.The present study conducted wave flume experiments on silt and silty fine sand beds with varying particle compositions.Furthermore,a comprehensive analysis of the differences and underlying reasons for liquefaction behavior in two different types of soil was conducted from both macroscopic and microscopic perspectives.The experimental results indicate that the silt bed necessitates a lower wave load intensity to attain the liquefaction state in comparison to the silty fine sand bed.Additionally,the duration and development depth of liquefaction are greater in the silt bed.The dissimilarity in liquefaction behavior between the two types of soil can be attributed to the variation in their permeability and plastic deformation capacity.The permeability coefficient and compression modulus of silt are lower than those of silty fine sand.Consequently,silt is more prone to the accumulation of pore pressure and subsequent liquefaction under external loading.Prior research has demonstrated that silt beds with varying consolidation degrees exhibit distinct initial failure modes.Specifically,a dense bed undergoes shear failure,whereas a loose bed experiences initial liquefaction failure.This study utilized discrete element simulation to examine the microscopic mechanisms that underlie this phenomenon.

基于Hadoop的离线电商数据分析系统的设计与实现

随着互联网的普及和人们不断增长的消费需求,电子商务平台已成为人们生活不可或缺的一部分,由此产生大量数据。基于Hadoop,设计并实现一个离线电商数据分析系统,从系统架构、数据采集、传输、存储、分析处理和可视化等方面作了介绍,系统具有一定的数据分析能力,能将原本无用的数据进行整理利用,发挥数据本身的价值。

Three-dimensional numerical simulation of flow in trapezoidal cutthroat flumes based on FLOW-3D

To solve the common problem of flumes flowmeasurement accuracy without sacrificing water head, a new type of trapezoidal cutthroat flume to measure the discharge in terminal trapezoidal channels is presented.Using the computational fluid dynamic method, threedimensional flow fields in trapezoidal cutthroat flumes were simulated using the RNG k-ε three-dimensional turbulence model along with the Tru VOF technique.Simulations were performed for 12 working conditions,with discharges up to 0.075 m3$s–1 to determine hydraulic performance. Experimental data for the trapezoidal cutthroat flume in terminal trapezoidal channel were also obtained to validate the simulation results. Velocity distribution of the flume obtained from simulation analyses were compared with observed results based on timeaveraged flow field and comparison yielded a solid agreement between results from the two methods, with relative error below 10%. The results indicated that the Froude number and the longitudinal average velocity increased along the convergence section and decreased in the divergent section. In the upper throat, the Froude number was less than 0.5, which meets the water measurement requirement, and the critical flow appeared near the throat section. The maximum water head loss of the trapezoidal cutthroat flume was less than 9% of the total head, compared to the rectangular cutthroat flume,and head loss of trapezoidal cutthroat flume was significantly less. Regression models developed for upstream depth versus discharge under different working conditions were satisfactory, with a relative error of less than 2.06%, which meets the common requirements of flow measurement in irrigation areas. It was concluded that trapezoidal cutthroat flumes can improve flow-measurement accuracy without sacrificing water head.

基于改进的Flume实时数据采集系统应用研究

通过对Apache Flume日志采集系统的实时数据采集方案进行比较,并针对Flume现有的实时数据采集方案进行分析,提出了一种基于Flume改进的实时数据采集方案。实验证明,改进后的实时数据采集方案具有可行性。

基于Spark Streaming的气象自动站实时流处理与存储系统

在当前大数据技术蓬勃发展的时代,人们对气象数据的实时处理、数据质量、数据存储及大规模查询等要求也越来越高。针对现有气象自动站数据业务落地环节多,任务处理耦合紧但系统部署分散等问题,文中基于Spark Streaming的流式计算框架,研究使用Flume解析收集自动站原始数据,在Spark Streaming中设计融入自动站数据质控算法,最终通过对分布式数据库存储的表设计,使气象自动站数据具备高效率、高质量、高可靠的应用服务能力。性能测试结果表明,基于Spark Streaming的气象自动站数据实时流处理与存储系统,数据从文件采集、解码、流处理至入库的全流程能够在秒级完成,TB级数据查询响应为毫秒级,加权查询为秒级,完全满足自动站数据业务应用需求,从而为进一步提高气象自动站数据质量与服务水平提供基础支撑。

Flume Experiment Investigation on Propagation Characteristics of Tidal Bore in A Curved Channel

Tidal bore is a special and intensive form of flow movement induced by tidal effect in estuary areas, which has complex characteristics of profile, propagation and flow velocity. Although it has been widely studied for the generation mechanism, propagation features and influencing factors, the curved channel will complicate the characteristics of tidal bore propagation, which need further investigation compared with straight channel. In this study, the flume experiments for both undular and breaking bores’ propagation in curved channel are performed to measure the freesurface elevation and flow velocity by ultrasonic sensors and ADV respectively. The propagation characteristics,including tidal bore height, cross-section surface gradient, tidal bore propagation celerity, and flow velocity are obtained for both sides of the curved channel. And three bore intensities are set for each type of tidal bores. The freesurface gradients are consistently enlarged in high-curvature section for undular and breaking bores, but have distinct behaviors in low-curvature section. The spatial distributions of tidal bore propagation celerity and flow velocity are compared between concave and convex banks. This work will provide experimental reference for engineering design of beach and seawall protection, erosion reduction and siltation promotion in estuary areas with the existence of tidal bores.

Modeling of breaching parameters for debris flow dams

The debris flow dam is a common type of barrier dams,which shows significant differences from other types of barrier dam such as landslide dam,moraine dam in their formation processes,dam body shapes,and internal compositions.The basic breaching parameters such as flood peak discharge are vital indicators of risk assessment.In this study,we elucidated the failure process of the debris flow dam through the flume experiment,and built the calculation equation of the breaching parameters by selecting critical factors.The result shows that the overtopping failure process of the debris flow dam is capable of forming significantly retrogressive scarps,and the failure process experiences three stages,the formation of the retrogressive scarp,the erosion of the retrogressive scarp,and the decline of the retrogressive scarp.Five factors used for establishing the calculation equations for peak discharge(Qp),final width(Wb)of the breach,and duration(T)of the debris flow dam failure are dam height(h),reservoir capacity(V),the fine grain content(P0.075)of the soil,the nonuniformity coefficient(Cu)of the soil,and the upper limit grain size(D90)of the soil,respectively.In the three equations,the correlation coefficients between Qp,Wb,T and the five factors were 0.86,0.70,0.63,respectively.The equations still need to be modified and verified in actual cases.

Characteristics of debris flow impact on a double-row slit dam

To accurately predict impact loads can ensure the safe operation of debris flow control projects.The instantaneous impact process is usually considered in the calculation of the debris flow impact force;however,the redistribution of an impact load after structural regulation is unclear.In this study we deduced the theoretical calculation of a debris flow impact on a double-row slit dam,and carried out a verification experiment on the debris flow impact.The calculation model considers the influence of the debris flow properties,dam arrangement and pile material.The results show that the impact force of the debris flow is obviously affected by the bulk density.When the bulk density is 21 kg/m^(3),the maximum impact force on the pile dam is 1.15 times that when the bulk density is 15 kg/m^(3),but the time it takes for the debris flow to pass through the dam body is reduced by 60%.The larger the relative pile spacing,the more sufficient the flow space and the lower the maximum impact force.The maximum impact force of relative pile spacing of 0.8 is 12%less than that of elative pile spacing of 0.5.The horizontal distribution of the impact force in the mud depth range is parabolic.The maximum impact force on the centre pier is 1.3 times that of a side pier,and the maximum impact force on the dam body appears at the top of the mud depth range.From the vertical distribution of the impact force,the maximum impact force at the highest mud mark is approximately 70%of that of the bottom.With the increase in the relative pile spacing,the longitudinal maximum impact force distribution first decreases and then increases.

基于Flume和Flink的轨道交通智能运维数据接收解析系统

随着轨道交通智能运维技术的不断发展,越来越多的车载设备将数据通过车载无线通信设备下发至地面服务器进行存储、分析以及可视化显示。通过研究相关大数据技术,提出一种基于日志收集框架Flume和流数据处理引擎Flink的智能运维数据接收解析系统,可在高吞吐量的基础上保证稳定性与实时性。

基于时序数据的卷烟工程试制平台工业大数据采集设计探索

利用OPC接口采集某工业企业的工程试制平台内设备产生的实时数据,并通过大数据组件Flume将数据归档至OpenTSDB时序数据库,从而完成了针对工程试制平台日常生产的过程数据采集和基于大数据平台的数据归档。该数据采集设计方案为上游的工业制造数据分析和实时监控系统提供了统一粒度的基础数据,实现了卷烟生产中工程试制平台与工业数据平台在数据层完整协同的设计目标。

模拟水流环境中抗生素的行为特征与归宿

利用香港理工大学水动力学实验室的大型流动水槽(FLUME)模拟动态水流环境,研究了4种不同类型的抗生素(红霉素、罗红霉素、氧氟沙星和磺胺甲噁唑)在动态和静态水体与沉积物之间的交换与配分,初步探明了抗生素类药物在河流环境中的行为及归宿.结果显示:在水流环境下,抗生素被迅速吸附到水体中悬浮的颗粒物和表层沉积物中,并可通过剪切力作用被吸附到次表层沉积物中.而在静止水体中,仅有少量的抗生素被吸附到表层沉积物中.氧氟沙星显示出强的吸附特性(DT50≥22 d),具有高的吸附系数(Kd),而磺胺甲噁唑的吸附能力较弱.在FLUME系统中,氧氟沙星具有适中的持久性,其他3种药物显示出弱的持久性;而在静止系统中,4种药物均显示出适中的持久性.抗生素在水流环境中的持久性要低于静止环境.颗粒物的吸附与自身的代谢是抗生素在水流环境中的主要归宿.

基于Hadoop与Storm的日志实时处理系统研究

日志数据记录着丰富的信息,具有较高的实用价值,但在当今大数据时代环境下,数据量的陡增为日志数据的处理带来了挑战.为了有效地解决海量日志数据处理面临的瓶颈问题,本文整合Hadoop和Storm分布式框架,构建一种融合了实时计算与离线计算的分布式日志实时处理系统.系统架构由数据服务层、业务逻辑层和Web展示层组成,数据服务层使用Flume实时采集日志数据,并分别采用Kafka与HBase完成实时日志流数据的缓冲和系统数据的持久化存储;业务逻辑层利用Storm对实时日志流数据进行实时分析,并使用Hadoop的计算引擎MapReduce结合数据挖掘技术完成对海量历史日志数据的离线分析,离线分析的结果为实时分析提供支持、参考;Web展示层负责日志数据及其分析结果的展示.实验结果表明,系统能有效地解决日志数据的采集存储、实时日志流数据的实时分析和历史日志数据的离线分析等问题,并成功地融合了Hadoop与Storm各自的优势,为日志数据的采集和分析系统的构建提供新的技术参考.

基于Flume的分布式日志采集分析系统设计与实现

互联网的迅猛发展以及大数据相关技术的兴起,使得日志数据的规模日益剧增,给已有的日志采集及分析系统带来了挑战;同时针对互联网安全的攻击活动也层出不穷,如何在海量的日志中及时提取有效的信息以为企业安全提供信息支撑,也给现有的分析系统提出了新的需求。本文结合使用Flume、Elasticsearch以及Kibana等技术手段提出了一种分布式的日志采集分析系统,从系统设计和架构等方面提出了新的解决思路,并针对Nginx的访问日志进行了实时采集和分析及展示,完成了原型系统的实现,实验证明,基于Flume的分布式的日志采集分析系统方案具有可行性。

基于Flume的MySQL数据自动收集系统

针对分布式系统中、不同数据中心之间的数据收集,同时解决将数据由关系型数据库交换到非关系型数据库的问题,提出一种基于Flume的MySQL数据库数据自动收集系统。为了符合现实中的生产环境,该系统采用的是一种星型拓扑结构。系统可以自动查询给定的MySQL数据库表,自动检测表中的数据更新,实现自动增量传输,并对原始数据进行封装、解析,最终将数据存储到非关系型数据库HBase中。在测试中,系统中每台机器的平均传输速度可达到1 111 kb/s,系统总的平均传输速度可以达到3 333 kb/s,并且保证了数据的完整性,实现了可靠高效传输的目标。

基于Storm的实时用户日志管理系统

研究并开发了具有实时响应的大数据用户日志管理分析平台。采用Storm流式计算框架,完成用户日志分析的实时处理平台架构;采用Elastic Search作为实时搜索引擎,完成实时响应处理;使用Flume、Kafka技术完成海量日志采集、聚合和传输等功能。以某城市监狱管理系统的用户日志信息为实验数据,系统取得了良好的实验效果。为同时处理海量数据和实时响应用户请求的系统提供了可行性解决方案。

基于通信运营商数据的大数据实时流处理系统

本文利用流式数据处理框架探索了一种新的基于运营商实时大数据业务系统构建模式。首先,在充分研究了业内实时流式处理技术的发展以及运营商本身实时数据源的特点之后,确定以Flume作为实时采集和分流组件,Kafka作为缓存和多模块通信组件,以Spark Streaming的分布式结构作为数据ETL集群;然后,利用该系统进行了重点区域的人流实时监控的业务,在实施过程中为了提供毫秒级的数据结果流查询能力,采用了Redis组件提供基于内存的Key-Value引擎;最终,通过流式数据处理效率的对比和实时监控的人流效果,我们验证了了这种新的技术架构针对运营商CS域和PS域数据实时处理需求的可行性,结果表明,新的实时业务架构能更加有效的提高从实时采集到业务触发的运行效率,并且为公安部门在重大节假日的区域级人流监控、预警和疏导提供了技术保障。

基于大数据车辆模拟运行及日志收集仿真研究

中国汽车保有量呈井喷式上涨,需要面对越来越多的车辆违法行为,如超速行驶、遮挡牌照、套牌、占道等等。针对这些问题,传统的车辆稽查方式显得力不从心,存在数据单一、实时性差、离线分析为主等缺点。为减轻交管部门的车辆稽查压力,构建了以大数据实时分析技术为基础的车辆模拟运行及日志收集仿真模拟系统。该仿真模拟系统采用Flume、Hadoop框架和Redis集群,参照城市街道布局设计,不仅对海量车辆的超高并发度运行和城市交通进行了模拟,还对车辆日志信息进行了实时收集。以此为依据,构建的车辆模拟运行场景,可以有效地提高车辆稽查效率。该系统为数据量较大的行业提供了一种解决问题的思路,用同样的方法可以为城市旅游、城市产业分布等进行场景模拟,进而调整旅游服务、产业调整等方案。

基于Apache Flume的分布式日志收集系统设计与实现

基于Apache Flume的开源日志收集系统,设计并实现了一个适用于异构网站平台间的分布式日志收集系统。该系统将多个网站的日志集中存储,为进行下一步的日志分析提供了数据支持,也为云平台下的网站日志收集方案提供了参考依据。

基于Apache Flume的大数据日志收集系统

在信息化及大数据时代下,各种应用程序的分散及生成的日志信息成爆炸式增长,怎么有效的收集这些日志,并对它进行在线和离线分析,已经成为这领域的热点。我们在现有的Apache Flume、Hadoop等框架下,设计实现了一种大数据日志收集系统。