Constructing a raster-based spatio-temporal hierarchical data model for marine fisheries application

Marine information has been increasing quickly. The traditional database technologies have disadvantages in manipulating large amounts of marine information which relates to the position in 3-D with the time. Recently, greater emphasis has been placed on GIS （geographical information system）to deal with the marine information. The GIS has shown great success for terrestrial applications in the last decades, but its use in marine fields has been far more restricted. One of the main reasons is that most of the GIS systems or their data models are designed for land applications. They cannot do well with the nature of the marine environment and for the marine information. And this becomes a fundamental challenge to the traditional GIS and its data structure. This work designed a data model, the raster-based spatio-temporal hierarchical data model （RSHDM）, for the marine information system, or for the knowledge discovery fi＇om spatio-temporal data, which bases itself on the nature of the marine data and overcomes the shortages of the current spatio-temporal models when they are used in the field. As an experiment, the marine fishery data warehouse （FDW） for marine fishery management was set up, which was based on the RSHDM. The experiment proved that the RSHDM can do well with the data and can extract easily the aggregations that the management needs at different levels.

Quality control of marine big data——a case study of real-time observation station data in Qingdao

Offshore waters provide resources for human beings,while on the other hand,threaten them because of marine disasters.Ocean stations are part of offshore observation networks,and the quality of their data is of great significance for exploiting and protecting the ocean.We used hourly mean wave height,temperature,and pressure real-time observation data taken in the Xiaomaidao station(in Qingdao,China)from June 1,2017,to May 31,2018,to explore the data quality using eight quality control methods,and to discriminate the most effective method for Xiaomaidao station.After using the eight quality control methods,the percentages of the mean wave height,temperature,and pressure data that passed the tests were 89.6%,88.3%,and 98.6%,respectively.With the marine disaster(wave alarm report)data,the values failed in the test mainly due to the influence of aging observation equipment and missing data transmissions.The mean wave height is often affected by dynamic marine disasters,so the continuity test method is not effective.The correlation test with other related parameters would be more useful for the mean wave height.

Study on Growth Laws of Marine Science and Technology Talents Based on the Perspective of Big Data

Based on the perspective of big data,the growth characteristics of marine science and technology talents were analyzed,and the growth of marine science and technology talents was divided into five periods:study period,adaptation period,growth period,promotion period and stability period.Moreover,some suggestions for the training of marine science and technology talents were proposed from the aspects of students,families,schools and society.

Hotshots of Spatio-temporal Behavior of Chinese Residents in the Context of Big Data:Visual Analysis Based on CiteSpace

By using CiteSpace software to create a knowledge map of authors,institutions and keywords,the literature on the spatio-temporal behavior of Chinese residents based on big data in the architectural planning discipline published in the China Academic Network Publishing Database(CNKI)was analyzed and discussed.It is found that there was a lack of communication and cooperation among research institutions and scholars;the research hotspots involved four main areas,including“application in tourism research”,“application in traffic travel research”,“application in work-housing relationship research”,and“application in personal family life research”.

A Marine Environment Early Warning Algorithm Based on Marine Data Sampled by Multiple Underwater Gliders

This study analyzes and summarizes seven main characteristics of the marine data sampled by multiple underwater gliders. These characteristics such as the big data volume and data sparseness make it extremely difficult to do some meaningful applications like early warning of marine environment. In order to make full use of the sea trial data, this paper gives the definition of two types of marine data cube which can integrate the big marine data sampled by multiple underwater gliders along saw-tooth paths, and proposes a data fitting algorithm based on time extraction and space compression(DFTS) to construct the temperature and conductivity data cubes. This research also presents an early warning algorithm based on data cube(EWDC) to realize the early warning of a new sampled data file.Experiments results show that the proposed methods are reasonable and effective. Our work is the first study to do some realistic applications on the data sampled by multiple underwater vehicles, and it provides a research framework for processing and analyzing the big marine data oriented to the applications of underwater gliders.

STGI:a spatio-temporal grid index model for marine big data

Marine big data are characterized by a large amount and complex structures,which bring great challenges to data management and retrieval.Based on the GeoSOT Grid Code and the composite index structure of the MongoDB database,this paper proposes a spatio-temporal grid index model(STGI)for efficient optimized query of marine big data.A spatio-temporal secondary index is created on the spatial code and time code columns to build a composite index in the MongoDB database used for the storage of massive marine data.Multiple comparative experiments demonstrate that the retrieval efficiency adopting the STGI approach is increased by more than two to three times compared with other index models.Through theoretical analysis and experimental verification,the conclusion could be achieved that the STGI model is quite suitable for retrieving large-scale spatial data with low time frequency,such as marine big data.

Construction of Smart City Spatio-Temporal Information Cloud Platform in Weifang,China

On the basis of the digital Weifang geospatial framework,Smart Weifang spatio-temporal information cloud platform(WFCP)integrated legal person information,population,place name and address data,macroeconomic data and so on.And it also expanded the data contents,such as the indoor and outdoor data,the overground and underground data,panoramic data and real data.It also introduced the contents of historical geographical information in different periods and real-time location information,address information of sensing equipment,real-time perception and interpreting information.It has overcome the difficulties of real-time access of Internet of Things(IoT)perception,multi-node collaboration,64-bit support,cluster deployment and has the characteristics of spatio-temporal management,ondemand service,large data analysis and micro-service architecture.It built spatio-temporal information big data center and spatio-temporal information cloud platform,realized the convergence and management of the distributed big data,deeply applied for land,transportation,environmental protection,police and subdistrict five areas,by supporting the integrated application of multi-source information and supporting intelligent deep application.In the aspect of hardware environment construction,according to the top-level design and unified arrangement of Smart Weifang,the WFCP was migrated to Weifang cloud computing center,to achieve the on-demand computing resources and dynamic scheduling load-based computing resources,to support the generalizing load map application.

区域级智慧海洋大数据中心架构规划及建设实践

智慧海洋大数据中心是智慧海洋工程建设的核心载体和体系中枢,其区域中心的规划和建设对于加快各省海洋强省建设、支撑地方数字政府建设、推动国家智慧海洋工程建设具有重要的支撑意义。本文基于推进重点领域突破的实际需求,提出一种区域级智慧海洋大数据中心的架构设计,阐述了其总体框架、技术架构和数据架构,并给出了实现的研究案例。基于架构设计的浙江省智慧海洋大数据中心提供了数据资源、能力支撑、综合展示、开放创新、应用服务等层面的关键技术,构建了浙江省海洋大数据安全开放的应用环境和自发主动、高效规范的共享模式,形成了国家首个省级海洋大数据中心示范效应和全国可复制可推广的建设经验。智慧海洋大数据中心作为新型信息化平台,将促进海洋大数据资源的共享应用和潜能释放,聚焦数据共享、应用生态、迭代升级和创新驱动,为海洋数据资源全生命周期管理共享和运营服务提供完整解决方案。

海洋科学数据汇聚共享服务平台建设

【应用背景】海洋数据是认识海洋、经略海洋的基础,随着海洋观测和探测技术的快速发展,海洋数据呈爆炸式增长,数据的开放共享需求日益迫切。【目的】为了实现多源海洋科学数据的整合,打通数据存储孤岛,建立统一的元数据标准和数据共享链路,【方法】本文构建了一个海洋科学数据汇聚共享服务平台,提供海洋数据汇交、审核、注册、存储、检索和共享全过程管理的一体化、在线化和流程化服务,为不同用户提供公平渠道获取数据、信息、知识和技术;提供DOI/CSTR自动注册服务,加强数据知识产权保护,保护数据所有者权益。【结果】平台以数据规范为牵引,将数据存储规范、管理流程规范、元数据标准纳入系统设计中,实现数据统一编码、统一管理、统一存储、统一发布的“四统一”。【结论】通过平台的应用推广,能够更好地推进海洋科学数据高效共享和利用,充分挖掘海洋信息资源的潜在价值,高质量支撑海洋科学研究。

海洋地质信息化建设研究进展与展望

回顾了海洋地质信息化建设历程,介绍了海洋地质信息化的顶层设计框架,归纳总结了20年来海洋地质信息化建设在海洋地质信息技术标准、网络体系、数据资源建设、数据共享与信息服务、海洋地质调查全流程信息化等方面取得的成果,认为海洋地质信息化成果丰硕,为海洋地质工作的现代化打下了坚实基础,发挥了较好的支撑作用。分析了当前海洋地质信息化存在的人员融合力度不足、感知设备不足、对成果的快速产出支撑力度不足、大数据分析和人工智能技术应用不足等问题,提出了海洋地质信息化工作今后应加强观念转变、落实行动促进人员深度融合、加强感知-计算-分析能力、拓展全流程软件生态、推进大数据分析、进一步加强人工智能在海洋地质调查研究的应用等。

海洋经济大数据智能应用系统建设研究

为解决大数据时代背景下现有海洋经济管理决策信息化建设存在的大数据资源汇聚融合不充分、存储计算架构难以满足大数据管理应用需要、大数据挖掘与智能分析决策能力不足等问题,在系统分析海洋经济管理决策业务支撑需求的基础上,提出基于大数据分析处理框架、空间大数据挖掘分析、GIS可视化等多种信息技术,建设海洋经济大数据智能应用系统的思路。详细阐述了系统总体架构、技术架构和数据架构设计,以及海洋经济形势分析、分析评估、大数据指数、涉海企业运行分析、经济地图等功能建设,为沿海地区海洋经济宏观、中观、微观管理提供多层次应用服务支撑,推动海洋经济管理向大数据科学决策转变。

基于腐蚀大数据的碳钢海洋大气腐蚀行为

随着“一带一路”和“海洋强国”等重大战略的深入推进,沿海地区及海上重大装备设施面临着空前严峻的海洋环境挑战。深入研究海洋大气腐蚀行为,对于延缓腐蚀过程、确保设施安全运行以及防止重大安全事故和经济损失具有至关重要的意义。以Q235碳钢作为研究对象,基于腐蚀大数据技术,通过周期浸泡加速试验获取实时连续的腐蚀数据,研究环境因子Cl^(−)、HSO_(3)^(−)及pH值对碳钢海洋大气腐蚀行为的影响。结果表明:随着Cl^(−)、HSO_(3)^(−)浓度的增加或pH值的降低,腐蚀累积量呈不断上升的趋势,腐蚀大数据采集结果与碳钢试样腐蚀形貌及锈层分析结果一致。随试验的进行,当Cl^(−)浓度较低时,锈层朝着更加稳定的方向发展;Cl^(−)浓度较高时,锈层稳定性不断下降。0.05%HSO_(3)^(−)浓度下,腐蚀始终以相对较低的速率匀速进行;浓度为0.1%时,腐蚀先是匀速进行,但由于腐蚀后期形成了具有一定保护性的锈层,腐蚀速率有所下降;当浓度达到0.2%时,腐蚀速率在试验后期反而加快,锈层难以维持相对稳定的状态。试验环境pH=5或3时,整个试验周期的腐蚀速率整体呈下降趋势;pH=1时,腐蚀速率呈持续上升趋势。腐蚀大数据方法与传统挂片法研究结果具有较高的一致性,验证了腐蚀大数据手段对于环境腐蚀研究的可靠性,研究结果可为后续开展数据分析与挖掘工作奠定基础。

大数据驱动海洋环境跨域治理的内在机理及实现机制

跨域合作治理是实现海洋环境有效治理的基本路径。在政府数字化转型的趋势下,以互联网、云计算、大数据等为代表的新兴技术为真正落实海洋环境跨域治理带来了新的契机。通过构建“结构—过程—功能”逻辑架构对大数据赋能海洋环境跨域治理的流程进行剖析发现,大数据驱动海洋环境跨域治理形成了“结构嵌入—过程迭变—功能阐发”的内在机理。研究提出,未来要基于大数据的优势,构建海洋环境跨域治理的协同预测决策机制、多元主体互动机制、跨域利益共享机制以及权力责任落实机制,从而实现区域海洋环境的整体有效治理。

海洋大数据标准化现状与对策研究

文中介绍了海洋大数据概念,分析了海洋大数据标准化组织、政策要求、平台建设、标准体系、标准研制和试点示范现状;指出存在标准不统一导致海洋大数据汇聚和管理应用困难、标准缺失较多等问题;提出完善海洋大数据标准化政策措施和工作机制,开展海洋大数据标准化研究和建立健全标准体系,开展海洋大数据标准化相关试点示范等对策建议。

HybridTune: Spatio-Temporal Performance Data Correlation for Performance Diagnosis of Big Data Systems

With tremendous growing interests in Big Data, the performance improvement of Big Data systems becomes more and more important. Among many steps, the first one is to analyze and diagnose performance bottlenecks of the Big Data systems. Currently, there are two major solutions. One is the pure data-driven diagnosis approach, which may be very time-consuming;the other is the rule-based analysis method, which usually requires prior knowledge. For Big Data applications like Spark workloads, we observe that the tasks in the same stages normally execute the same or similar codes on each data partition. On basis of the stage similarity and distributed characteristics of Big Data systems, we analyze the behaviors of the Big Data applications in terms of both system and micro-architectural metrics of each stage. Furthermore, for different performance problems, we propose a hybrid approach that combines prior rules and machine learning algorithms to detect performance anomalies, such as straggler tasks, task assignment imbalance, data skew, abnormal nodes and outlier metrics. Following this methodology, we design and implement a lightweight, extensible tool, named HybridTune, and measure the overhead and anomaly detection effectiveness of HybridTune using the BigDataBench benchmarks. Our experiments show that the overhead of HybridTune is only 5%, and the accuracy of outlier detection algorithm reaches up to 93%. Finally, we report several use cases diagnosing Spark and Hadoop workloads using BigDataBench, which demonstrates the potential use of HybridTune.

海洋大数据的内涵、现状与发展趋势展望

海洋数据是国家重要的基础性战略资源,海洋大数据融合应用发展对于实施海洋强国战略意义重大。本文从数据、技术和应用3个角度分析归纳了海洋大数据的内涵、外延和特征,提出了低质性、碎片化和隐匿性3个海洋大数据要解决的关键问题。从海洋大数据资源体系、海洋信息通信网、海洋大数据处理分析、海洋大数据存储管理和共享服务,以及海洋大数据应用研发等方面总结了海洋大数据发展能力现状和趋势,探讨了当前国家海洋大数据中心发展需求,架构海洋大数据国内外总体布局,从数据资源、关键技术、共享服务、开放创新、融合应用和国际合作等6个方面,研究提出海洋大数据重点发展方向,以期为国家、行业和地方海洋大数据发展提供参考。

国家海洋科学数据共享服务体系构建与应用服务

海洋科学数据共享是推动海洋科学发展、提升数据价值的关键。本文回顾了国家海洋科学数据中心的发展历程,从数据收集与整合、数据保存与管理、关键技术攻关与产品研制、系统平台建设、共享服务成效与推广应用等方面总结了国家海洋科学数据中心在海洋科学数据共享领域的研究实践及成果。文章最后提出在大数据时代,数据密集型的科学研究已成为新的科研范式,海洋科学大数据的交叉融合、挖掘分析与综合应用,将给海洋科学数据共享带来新的机遇与挑战。

海洋极端天气现象预测方法研究进展

海洋极端天气对沿海地区影响重大。研究人员利用海洋大数据,结合深度学习算法,在海洋极端天气现象预测方面取得了重要进展。本文首先以典型的多尺度海洋极端天气现象——厄尔尼诺、台风及短临降雨为例,介绍了近年来主流的海洋极端天气现象预测算法,即基于模式计算的方法和基于人工智能的算法。然后分析了海洋极端天气现象智能预测的挑战和机遇,详细总结了各类方法的研究进展,并且通过数据和实验讨论了现有算法的优点和不足。最后展望了基于海洋大数据的海洋极端天气现象智能预测的发展方向。

群体学习和隐私保护树在海洋盐度反演中的应用

为在不违反隐私的情况下促进全球多源海洋数据集成与联合使用,该文引入群体学习(Swarm Leanring)和隐私保护树(Secure Boost)用于数据隐私整合和树形模型训练,以海洋盐度反演作为实践场景,提出SLSB-LightGBM模型,并在全球4片海域内进行测试。使用各区隐私数据分别构建SLSB-LightGBM全局模型和SLSB-LightGBM迁移模型,基于本地学习思想构建4个单区LightGBM模型作为对照,结果表明:单区模型在复杂海域的盐度反演效果较差,且适用区域局限性明显;SLSB-LightGBM模型在1 PSU和0.8 PSU条件下精度分别高于84%和76%,均方误差(MSE)和平均绝对误差(MAE)均小于0.58,同时泛化性明显高于本地学习,在盐度分布复杂和数据量小的情况下仍能保证模型准确度,而且SLSB-LightGBM模型在设计和执行过程中完全保证了隐私性,在海洋领域的初次尝试表现卓越,可为探索复杂场景的海洋数据隐私联合使用提供新的方法。

支持大数据的参数自适应支持向量回归方法

为解决大数据条件下的高效精准预测问题,提出一种基于均值漂移聚类与海洋捕食者算法的参数自适应支持向量回归方法。将大数据样本划分为训练组、验证组和测试组;采用均值漂移聚类算法处理训练组得到聚类中心;设定支持向量回归(SVR)参数,随机生成多个SVR参数组;基于参数组和聚类中心,采用支持向量回归算法对验证组样本进行预测以得到预测精度,然后采用海洋捕食者算法更新SVR参数组,循环本步骤直到满足截止条件,从而获得最优SVR参数组;基于该最优参数组,用SVR获得测试组的预测精度。与高度类似方法在预测精度、稳定性和数据损失等方面进行比较,验证了该方法的可行性和有效性。