期刊文献+
共找到1,253篇文章
< 1 2 63 >
每页显示 20 50 100
Improvement Design for Distributed Real-Time Stream Processing Systems 被引量:4
1
作者 Wei Jiang Liu-Gen Xu +1 位作者 Hai-Bo Hu Yue Ma 《Journal of Electronic Science and Technology》 CAS CSCD 2019年第1期3-12,共10页
In the era of Big Data, typical architecture of distributed real-time stream processing systems is the combination of Flume, Kafka, and Storm. As a kind of distributed message system, Kafka has the characteristics of ... In the era of Big Data, typical architecture of distributed real-time stream processing systems is the combination of Flume, Kafka, and Storm. As a kind of distributed message system, Kafka has the characteristics of horizontal scalability and high throughput, which is manly deployed in many areas in order to address the problem of speed mismatch between message producers and consumers. When using Kafka, we need to quickly receive data sent by producers. In addition, we need to send data to consumers quickly. Therefore, the performance of Kafka is of critical importance to the performance of the whole stream processing system. In this paper, we propose the improved design of real-time stream processing systems, and focus on improving the Kafka's data loading process.We use Kafka cat to transfer data from the source to Kafka topic directly, which can reduce the network transmission. We also utilize the memory file system to accelerate the process of data loading, which can address the bottleneck and performance problems caused by disk I/O. Extensive experiments are conducted to evaluate the performance, which show the superiority of our improved design. 展开更多
关键词 Kafka Kafka CAT memory FILE SYSTEM MESSAGE QUEUE real-time stream processing SYSTEM
下载PDF
A Real-Time TCP Stream Reassembly Mechanism in High-Speed Network 被引量:3
2
作者 熊兵 陈晓苏 陈宁 《Journal of Southwest Jiaotong University(English Edition)》 2009年第3期185-191,共7页
With the continual growth of the variety and complexity of network crime means, the traditional packet feature matching cannot detect all kinds of intrusion behaviors completely. It is urgent to reassemble network str... With the continual growth of the variety and complexity of network crime means, the traditional packet feature matching cannot detect all kinds of intrusion behaviors completely. It is urgent to reassemble network stream to perform packet processing at a semantic level above the network layer. This paper presents an efficient TCP stream reassembly mechanism for real-time processing of high-speed network traffic. By analyzing the characteristics of network stream in high-speed network and TCP connection establishment process, several polices for designing the reassembly mechanism are built. Then, the reassembly implementation is elaborated in accordance with the policies. Finally, the reassembly mechanism is compared with the traditional reassembly mechanism by the network traffic captured in a typical gigabit gateway. Experiment results illustrate that the reassembly mechanism is efficient and can satisfy the real-time property requirement of traffic analysis system in high-speed network. 展开更多
关键词 TCP stream reassembly High-speed network real-time property Reassembly policy
下载PDF
Real-time Audio &Video Transmission System Based on Visible Light Communication 被引量:3
3
作者 Yingjie He Liwei Ding +1 位作者 Yuxian Gong Yongjin Wang 《Optics and Photonics Journal》 2013年第2期153-157,共5页
With the increasing popularity of solid sate lighting devices, Visible Light Communication (VLC) is globally recognized as an advanced and promising technology to realize short-range, high speed as well as large capac... With the increasing popularity of solid sate lighting devices, Visible Light Communication (VLC) is globally recognized as an advanced and promising technology to realize short-range, high speed as well as large capacity wireless data transmission. In this paper, we propose a prototype of real-time audio and video broadcast system using inexpensive commercially available light emitting diode (LED) lamps. Experimental results show that real-time high quality audio and video with the maximum distance of 3 m can be achieved through proper layout of LED sources and improvement of concentration effects. Lighting model within room environment is designed and simulated which indicates close relationship between layout of light sources and distribution of illuminance. 展开更多
关键词 VISIBLE LIGHT Communications LED real-time video and AUDIO BROADCAST System LIGHT Source Arrangement ILLUMINANCE Distribution
下载PDF
Sentiment Drift Detection and Analysis in Real Time Twitter Data Streams
4
作者 E.Susi A.P.Shanthi 《Computer Systems Science & Engineering》 SCIE EI 2023年第6期3231-3246,共16页
Handling sentiment drifts in real time twitter data streams are a challen-ging task while performing sentiment classifications,because of the changes that occur in the sentiments of twitter users,with respect to time.... Handling sentiment drifts in real time twitter data streams are a challen-ging task while performing sentiment classifications,because of the changes that occur in the sentiments of twitter users,with respect to time.The growing volume of tweets with sentiment drifts has led to the need for devising an adaptive approach to detect and handle this drift in real time.This work proposes an adap-tive learning algorithm-based framework,Twitter Sentiment Drift Analysis-Bidir-ectional Encoder Representations from Transformers(TSDA-BERT),which introduces a sentiment drift measure to detect drifts and a domain impact score to adaptively retrain the classification model with domain relevant data in real time.The framework also works on static data by converting them to data streams using the Kafka tool.The experiments conducted on real time and simulated tweets of sports,health care andfinancial topics show that the proposed system is able to detect sentiment drifts and maintain the performance of the classification model,with accuracies of 91%,87%and 90%,respectively.Though the results have been provided only for a few topics,as a proof of concept,this framework can be applied to detect sentiment drifts and perform sentiment classification on real time data streams of any topic. 展开更多
关键词 Sentiment drift sentiment classification big data BERT real time data streams TWITTER
下载PDF
Priority-Based Real-Time Stream Coding over Multi-Channel with Packet Erasures
5
作者 Yunkai Wei Qianyu Liu +1 位作者 Zhoucha Hu Yuming Mao 《China Communications》 SCIE CSCD 2017年第1期72-85,共14页
Multi-channel can be used to provide higher transmission ability to the bandwidth-intensive and delay-sensitive real-time streams. However, traditional channel capacity theories and coding schemes are seldom designed ... Multi-channel can be used to provide higher transmission ability to the bandwidth-intensive and delay-sensitive real-time streams. However, traditional channel capacity theories and coding schemes are seldom designed for the real-time streams with strict delay constraint, especially in multi-channel context. This paper considers a real-time stream system, where real-time messages with different importance should be transmitted through several packet erasure channels, and be decoded by the receiver within a fixed delay. Based on window erasure channels and i.i.d.(identically and independently distributed) erasure channels, we derive the Multi-channel Real-time Stream Transmission(MRST) capacity models for Symmetric Real-time(SR) streams and Asymmetric Real-time(AR) streams respectively. Moreover, for window erasures, a Maximum Equilibrium Intra-session Code(MEIC) is presented for SR and AR streams, and is shown able to asymptotically achieve the theoretical MRST capacity. For i.i.d. erasures, we propose an Adaptive Maximum Equilibrium Intra-session Code(AMEIC), and then prove AMEIC can closely approach the MRST transmission capacity. Finally, the performances of the proposed codes are verified by simulations. 展开更多
关键词 real-time stream MULTI-CHANNEL channel coding erasure model transmission capacity
下载PDF
符合Real-time CORBA规范的传感器网络监控系统设计 被引量:1
6
作者 魏云华 《计算机工程与设计》 CSCD 北大核心 2006年第23期4585-4588,4591,共5页
传感器网络在解决底层数据源和I/O设备的异构性、处理连续查询的实时性、服务质量自适应等方面提出了许多新的挑战。提出一种基于实时CORBA的传感器网络数据流处理框架,说明了符合实时中间件规范的数据流处理系统设计的方法和特点,通过... 传感器网络在解决底层数据源和I/O设备的异构性、处理连续查询的实时性、服务质量自适应等方面提出了许多新的挑战。提出一种基于实时CORBA的传感器网络数据流处理框架,说明了符合实时中间件规范的数据流处理系统设计的方法和特点,通过实验对该框架进行了性能测试,验证了其处理连续查询的实时性。 展开更多
关键词 实时中间件 数据流 连续查询 服务质量 传感器网络 实时CORBA
下载PDF
View Direction Adaptive 360 Degree Video Streaming System Based on Projected Area
7
作者 San Kim Jihyeok Yun +3 位作者 Bokyun Jo Ji Ho Kim Ho Gyun Chae Doug Young Suh 《Journal of Computer and Communications》 2018年第1期203-212,共10页
360 video streaming services over the network are becoming popular. In particular, it is easy to experience 360 video through the already popular smartphone. However, due to the nature of 360 video, it is difficult to... 360 video streaming services over the network are becoming popular. In particular, it is easy to experience 360 video through the already popular smartphone. However, due to the nature of 360 video, it is difficult to provide stable streaming service in general network environment because the size of data to send is larger than that of conventional video. Also, the real user's viewing area is very small compared to the sending amount. In this paper, we propose a system that can provide high quality 360 video streaming services to the users more efficiently in the cloud. In particular, we propose a streaming system focused on using a head mount display (HMD). 展开更多
关键词 360 PANORAMIC video Cloud SPLIT Image real-time streamING MPU
下载PDF
基于Spark Streaming流回归的煤矿瓦斯浓度实时预测 被引量:11
8
作者 吴海波 施式亮 念其锋 《中国安全生产科学技术》 CAS CSCD 北大核心 2017年第5期84-89,共6页
为了实时分析瓦斯监测流数据并对瓦斯浓度进行准确预测以实现瓦斯灾害实时预警,以实时流数据处理框架Spark Streaming构建基于流回归的瓦斯浓度实时预测系统。系统采用分布式流处理技术,可使基于回归算法的瓦斯浓度预测模型更新周期达... 为了实时分析瓦斯监测流数据并对瓦斯浓度进行准确预测以实现瓦斯灾害实时预警,以实时流数据处理框架Spark Streaming构建基于流回归的瓦斯浓度实时预测系统。系统采用分布式流处理技术,可使基于回归算法的瓦斯浓度预测模型更新周期达到秒级,提高了瓦斯浓度预测精度,满足流式大数据处理的实时性要求。实验表明:应用Spark Streaming流回归预测系统在采样周期为5s的瓦斯监测数据流上进行实时预测时,预测平均均方根误差随模型更新周期的缩短而减小,模型更新周期可达15s,且更新周期为45s时预测总均方根误差最小,既能保证预测精度,又能提高瓦斯灾害预警时效。 展开更多
关键词 监测数据 流数据 瓦斯浓度 SPARK streamING 流回归 实时预测 灾害预警
下载PDF
使用Spark Streaming的自适应实时DDoS检测和防御技术 被引量:10
9
作者 方峰 蔡志平 +2 位作者 肇启佳 林加润 朱明 《计算机科学与探索》 CSCD 北大核心 2016年第5期601-611,共11页
分布式拒绝服务(distributed denial of service,DDoS)攻击是重要的安全威胁,网络速度的不断提高给传统的检测方法带来了新的挑战。以Spark等为代表的大数据处理技术,给网络安全的高速检测带来了新的契机。提出了一种基于Spark Streamin... 分布式拒绝服务(distributed denial of service,DDoS)攻击是重要的安全威胁,网络速度的不断提高给传统的检测方法带来了新的挑战。以Spark等为代表的大数据处理技术,给网络安全的高速检测带来了新的契机。提出了一种基于Spark Streaming框架的自适应实时DDoS检测防御技术,通过对滑动窗口内源簇进行分组,并根据与各分组内源簇比例的偏差统计,检测出DDoS攻击流量。通过感知合法的网络流量,实现了对DDoS攻击的自适应快速检测和有效响应。实验结果表明,该技术可极大地提升检测能力,为保障网络服务性能和安全检测的可扩展性提供了一种可行的解决方案。 展开更多
关键词 DDOS检测 DDOS防御 实时检测 自适应检测 SPARK streamING
下载PDF
Spark Streaming动态资源分配策略 被引量:6
10
作者 刘备 谭新明 曹文彬 《计算机应用》 CSCD 北大核心 2017年第6期1574-1579,共6页
针对Spark Streaming作为混合大数据计算平台流处理组件时资源调整周期长和不能满足多应用多用户个性化需求的问题,提出了一种多应用下动态资源分配策略(DRAM)。该策略增加了应用全局变量来控制动态资源分配过程。首先,获取历史执行数... 针对Spark Streaming作为混合大数据计算平台流处理组件时资源调整周期长和不能满足多应用多用户个性化需求的问题,提出了一种多应用下动态资源分配策略(DRAM)。该策略增加了应用全局变量来控制动态资源分配过程。首先,获取历史执行数据反馈和应用全局变量;然后,进行资源增减计算;最后,进行资源增减执行。实验结果表明,所提策略能够有效调整应用资源配额,且在稳定数据流和不稳定数据流两种情况下,其处理延时相比原Spark平台的Streaming策略和Core策略都有所降低;同时该策略也能够提高集群资源利用率。 展开更多
关键词 SPARK 实时数据流 多应用 动态资源分配
下载PDF
基于Spark Streaming的实时数据采集分析系统设计 被引量:4
11
作者 党寿江 刘学 +1 位作者 王星凯 刘春梅 《网络新媒体技术》 2017年第5期48-53,共6页
大数据量的实时数据分析系统,需要快速的处理和响应。为了实现实时数据分析,本文设计了基于Spark Streaming的实时数据采集分析系统,并对有状态计算操作的基数计算的精确计算方法和估算方法进行了比较分析。实验表明,基于HyperLogLog++... 大数据量的实时数据分析系统,需要快速的处理和响应。为了实现实时数据分析,本文设计了基于Spark Streaming的实时数据采集分析系统,并对有状态计算操作的基数计算的精确计算方法和估算方法进行了比较分析。实验表明,基于HyperLogLog++的基数估算方法在处理时间和存储占用空间上有明显优势,而计算偏差基本可以忽略不计,更适于大数据的基数估算。 展开更多
关键词 基数计算 实时数据分析 Spark流式处理 不重复计数 HyperLogLog++
下载PDF
基于Spark Streaming的视频/图像流处理与新的性能评估方法 被引量:10
12
作者 黄文辉 冯瑞 《计算机工程与科学》 CSCD 北大核心 2015年第11期2055-2060,共6页
智能视频监控技术在公共安全、交通管理、智慧城市等方面有着广泛的运用前景,需求日益增长。随着摄像头安装的数量越来越多,采集的图像数据量越来越大,靠单台计算机处理已经远远不能满足需求了。分布式计算的兴起与发展为解决大规模的... 智能视频监控技术在公共安全、交通管理、智慧城市等方面有着广泛的运用前景,需求日益增长。随着摄像头安装的数量越来越多,采集的图像数据量越来越大,靠单台计算机处理已经远远不能满足需求了。分布式计算的兴起与发展为解决大规模的数据处理问题提供了很好的途径。使用一种基于Spark Streaming的视频/图像流处理的测试平台,阐述了平台的构成和工作流程,深入研究各个参数对集群性能的影响,创新性地提出了CPU时间占用率作为性能评估指标,与总的处理时间结合,更为全面反映集群性能和资源利用率。 展开更多
关键词 SPARK streamING CPU时间占用率 分布式 视频/图像处理
下载PDF
基于Spark Streaming的实时日志处理平台设计与实现 被引量:12
13
作者 薛瑞 朱晓民 《电信工程技术与标准化》 2015年第9期55-58,共4页
本文基于Hadoop Yarn平台,通过Spark Streaming流数据处理技术实时处理Web日志、系统日志、监控日志等,很好地解决了目前存在的问题,并使用HBase(Hadoop No SQL Database)作为数据库存储,更好地适应日志数据规模的增长。
关键词 HADOOP Sparkstreaming 实时日志处理系统
下载PDF
一个安全的CORBA音/视频流服务模型:SecStream 被引量:1
14
作者 冯名正 《软件学报》 EI CSCD 北大核心 2004年第11期1733-1740,共8页
通过对传统CORBA(common object request broker architecture)体系结构的扩展,OMG(Object Management Group)提出的CORBA音/视频流服务规范使得多媒体应用可以获得分布式对象计算在开放性、标准化等方面的优势.但该规范在安全方面有所... 通过对传统CORBA(common object request broker architecture)体系结构的扩展,OMG(Object Management Group)提出的CORBA音/视频流服务规范使得多媒体应用可以获得分布式对象计算在开放性、标准化等方面的优势.但该规范在安全方面有所欠缺,没有提供解决分布多媒体应用所面临的安全问题的完整方案.通过引入CORBA安全服务,并定义数据安全服务的概念,对原有流服务体系结构进行了安全扩展,提出了一个完善的安全流服务模型SecStream,对其进行设计和实现,并加以实践应用,验证了该模型可以灵活地提供多种安全保证,并具有较大的可扩展性. 展开更多
关键词 公共对象请求代理体系结构 服务质量 资源预留协议 实时流协议
下载PDF
基于Spark Streaming的明安图射电频谱日像仪实时数据处理 被引量:4
15
作者 卫守林 刘鹏翔 +3 位作者 王锋 邓辉 梁波 戴伟 《天文研究与技术》 CSCD 2017年第4期421-428,共8页
目前天文观测中对数据的实时处理需求越来越多,性能要求也越来越高,我国明安图射电频谱日像仪(Mingant U Sp Ectral Radioheliograph,MUSER)是同时以高时间、高空间和高频率分辨率对太阳进行射电频谱成像的设备。在低频部分的日常观测中... 目前天文观测中对数据的实时处理需求越来越多,性能要求也越来越高,我国明安图射电频谱日像仪(Mingant U Sp Ectral Radioheliograph,MUSER)是同时以高时间、高空间和高频率分辨率对太阳进行射电频谱成像的设备。在低频部分的日常观测中,包含了两方面的需求:(1)对历史数据的处理;(2)5秒钟抽样观测数据的处理。抽样观测数据需要实时处理,并在监控终端显示,数据处理过程包含了数据校验、修正、成图、洁化等多个步骤,传统的单机处理模式已无法满足大数据量下的实时性要求。因此,实时数据计算中,使用Spark Streaming流式计算这一新兴的分布式计算方法,设计了自定义的接收器,并将多个图形处理器节点加入到分布式集群中。通过实验对性能进行评估,结果证明基于内存的高速执行引擎的特点能显著提高性能。期待能通过实验进一步优化算法和配置,获得更好的结果,并最终运用到实际环境中。 展开更多
关键词 MUSER 射电天文 SPARK 流式计算 实时计算
下载PDF
基于Spark Streaming的实时流数据处理模型化研究与实现 被引量:2
16
作者 云惟英 苟宇 +1 位作者 王京 王丽莉 《测绘与空间地理信息》 2017年第S1期48-50,55,共4页
通过研究与分析,选取Spark Streaming技术实现对P实时流数据的处理.同时,研究出一套模型化的方式,实现动态装配软件的执行过程;并通过具体的实例展示了两者结合后,在数据处理的易用性、性能及吞吐量方面,都得到了大幅提升.
关键词 SPARK streamING 空间大数据 时实流数据
下载PDF
基于Oracle Streams的数据库实时备份技术研究 被引量:1
17
作者 成雅 毛宇光 《软件》 2011年第6期29-31,34,共4页
在数据库的故障恢复中,实现无数据丢失的数据库备份与恢复是极为重要的研究内容,运用数据库的实时备份,当有数据库操作时,可以实时地对数据库发生的改变进行备份,从而有效地防止数据丢失。基于Oracle Streams技术,提出了一个数据库的实... 在数据库的故障恢复中,实现无数据丢失的数据库备份与恢复是极为重要的研究内容,运用数据库的实时备份,当有数据库操作时,可以实时地对数据库发生的改变进行备份,从而有效地防止数据丢失。基于Oracle Streams技术,提出了一个数据库的实时备份方案。该方案利用Oracle Streams复制技术,配置捕获进程自动捕获数据库的所有改变,并将这些改变储存在高级队列中,传播至目标数据库,实现数据库的实时备份,恢复时从备份中读取这些改变信息,应用于目标数据库,可以得到一个处于一致性状态的无数据丢失的数据库。在Linux环境下技进行测试实验,结果表明本方案是可行的。 展开更多
关键词 计算机软件 实时备份 ORACLE streamS 逻辑改变记录
下载PDF
Turbo-Based DNW Algorithm for Compressed Video in VLC Domain 被引量:3
18
作者 LINGHe-fei LUZheng-ding ZOUFu-hao 《Wuhan University Journal of Natural Sciences》 EI CAS 2005年第1期297-302,共6页
In order to improve the robustness of the differential number watermarking (DNW) algorithm proposed by us before, we proposed turbo-based DNW (T-DNW) in which the turbo code was employed in the DNW algorithm. The turb... In order to improve the robustness of the differential number watermarking (DNW) algorithm proposed by us before, we proposed turbo-based DNW (T-DNW) in which the turbo code was employed in the DNW algorithm. The turbo code was used to encode the message prior to watermark embedding and decode the watermark posterior to watermark detection. From the analysis and experiments, the following conclusion could be drawn. The T-DNW algorithm has little higher computational complexity than DNW. And both algorithms have the same performance in terms of watermark visual quality impact. Furthermore, the T-DNW algorithm is much more robust against some common attack than DNW. Although the T-DNW algorithm sacrifices a half payload, we think the achievements are encouraging. 展开更多
关键词 real time video watermarking TURBO based DNW (T DNW) VLC domain
下载PDF
面向实时交互的P2P Streaming模型
19
作者 许成 蔡鸿明 姜丽红 《计算机工程》 CAS CSCD 北大核心 2008年第22期103-105,共3页
实施大规模分布式可视化操纵系统的一个重要瓶颈在于节点间海量数据的传输。P2P Streaming技术逐渐成为解决该瓶颈的可行方案之一。主流P2P Streaming模型由于实时性上的缺陷并不适于此类系统。该文通过引入服务质量的概念并采用启发式... 实施大规模分布式可视化操纵系统的一个重要瓶颈在于节点间海量数据的传输。P2P Streaming技术逐渐成为解决该瓶颈的可行方案之一。主流P2P Streaming模型由于实时性上的缺陷并不适于此类系统。该文通过引入服务质量的概念并采用启发式的算法,设计一种改良的P2P Streaming模型。仿真实验结果表明,新模型在平均延迟、首包到达时间等指标上优于现有模型。 展开更多
关键词 P2P streaming技术 实时交互 服务质量
下载PDF
RTP payload format for H.264/SVC scalable video coding 被引量:7
20
作者 WENGER Stephan HANNUKSELA Miska M. 《Journal of Zhejiang University-Science A(Applied Physics & Engineering)》 SCIE EI CAS CSCD 2006年第5期657-667,共11页
The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design ... The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC. Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL) and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, in- cluding the design criteria, use cases, signaling and payload structure. 展开更多
关键词 H.264 Advanced video CODING (AVC) Scalable video CODING (SVC) Scalability real-time transport protocol (RTP) Packetization
下载PDF
上一页 1 2 63 下一页 到第
使用帮助 返回顶部