大数据平台数据实时接出服务的技术实现

Analysis of Real-time Computing Monitoring Scheme for Big Data Platform

在DT时代,数据是宝贵的生产资料,新业务模式的创新离不开数据的支持,但不断扩大的数据规模也给大数据计算服务带来了极大的挑战。随着大数据平台服务能力的不断增强,下游应用系统对于数据获取方式和频率也在不断变化。传统的以离线文件传输提供数据的方式已不能满足下游系统的多样性需求,因此大数据平台需要对外部系统提供一种全新的数据实时接出服务功能,以满足各应用系统对于关键、敏感、核心业务数据信息的及时获取需求。数据接出服务作为大数据平台数据传输的接口,应满足客户端对获取数据服务的需求。数据接出服务采用RESTAPI的方式对外提供数据接出接口,客户端在经过安全验证后可以方便快捷地实现数据获取。

In the era of data information,data is a valuable means of production,and the innovation of new business model is inseparable from the support of data.However,the expanding data scale also brings great challenges to big data computing services.With the continuous enhancement of the service ability of big data platform,the data acquisition mode and frequency of downstream application system are also changing.The traditional way of providing data by off-line file transmission cannot meet the diversity needs of downstream systems.Therefore,the big data platform needs to provide a new real-time data out service function for external systems to meet the needs of key,sensitive and core business data information timely acquisition of application systems.As the data transmission interface of big data platform,data out service should meet the needs of client for data acquisition service.The data access service uses the rest API method to provide the data access interface to the external,and the client can easily and quickly pull the data after the security verification.