A New Efficient Obstacle Avoidance Control Method for Cars Based on Big Data and Just-in-Time Modeling

This paper provides a new obstacle avoidance control method for cars based on big data and just-in-time modeling. Just-in-time modeling is a new kind of data-driven control technique in the age of big data and is used in various real systems. The main property of the proposed method is that a gain and a control time which are parameters in the control input to avoid an encountered obstacle are computed from a database which includes a lot of driving data in various situations. Especially, the important advantage of the method is small computation time, and hence it realizes real-time obstacle avoidance control for cars. From some numerical simulations, it is showed that the new control method can make the car avoid various obstacles efficiently in comparison with the previous method.

Quality control of marine big data——a case study of real-time observation station data in Qingdao

Offshore waters provide resources for human beings,while on the other hand,threaten them because of marine disasters.Ocean stations are part of offshore observation networks,and the quality of their data is of great significance for exploiting and protecting the ocean.We used hourly mean wave height,temperature,and pressure real-time observation data taken in the Xiaomaidao station(in Qingdao,China)from June 1,2017,to May 31,2018,to explore the data quality using eight quality control methods,and to discriminate the most effective method for Xiaomaidao station.After using the eight quality control methods,the percentages of the mean wave height,temperature,and pressure data that passed the tests were 89.6%,88.3%,and 98.6%,respectively.With the marine disaster(wave alarm report)data,the values failed in the test mainly due to the influence of aging observation equipment and missing data transmissions.The mean wave height is often affected by dynamic marine disasters,so the continuity test method is not effective.The correlation test with other related parameters would be more useful for the mean wave height.

“Deep-time Digital Basin” Based on Big Data and Artificial Intelligence

1 Introduction Information technology has been playing an ever-increasing role in geoscience.Sphisicated database platforms are essential for geological data storage,analysis and exchange of Big Data(Feblowitz,2013;Zhang et al.,2016;Teng et al.,2016;Tian and Li,2018).The United States has built an information-sharing platform for state-owned scientific data as a national strategy.

Integrated Real-Time Big Data Stream Sentiment Analysis Service

Opinion (sentiment) analysis on big data streams from the constantly generated text streams on social media networks to hundreds of millions of online consumer reviews provides many organizations in every field with opportunities to discover valuable intelligence from the massive user generated text streams. However, the traditional content analysis frameworks are inefficient to handle the unprecedentedly big volume of unstructured text streams and the complexity of text analysis tasks for the real time opinion analysis on the big data streams. In this paper, we propose a parallel real time sentiment analysis system: Social Media Data Stream Sentiment Analysis Service (SMDSSAS) that performs multiple phases of sentiment analysis of social media text streams effectively in real time with two fully analytic opinion mining models to combat the scale of text data streams and the complexity of sentiment analysis processing on unstructured text streams. We propose two aspect based opinion mining models: Deterministic and Probabilistic sentiment models for a real time sentiment analysis on the user given topic related data streams. Experiments on the social media Twitter stream traffic captured during the pre-election weeks of the 2016 Presidential election for real-time analysis of public opinions toward two presidential candidates showed that the proposed system was able to predict correctly Donald Trump as the winner of the 2016 Presidential election. The cross validation results showed that the proposed sentiment models with the real-time streaming components in our proposed framework delivered effectively the analysis of the opinions on two presidential candidates with average 81% accuracy for the Deterministic model and 80% for the Probabilistic model, which are 1% - 22% improvements from the results of the existing literature.

Literature Review of Marketing theory based on Big Data

Since the concept of big data was proposed, the theory on big data is concerned by public, academics, market watchers, researcher and so on, people explore all aspects of the Big Data Time, more than in academic, it has an impact on all areas in marketing,we collect some papers and extract its viewpoints that involve the theory, methods in this article, we hope that it helps to do research on the theory of big data in the field of marketing.

Quantitative Expression of Paleogeographic Information Based on Big Data

Paleogeographic analysis accounts for an essential part of geological research,making important contributions in the reconstruction of depositional environments and tectonic evolution histories(Ingalls et al.,2016;Merdith et al.,2017),the prediction of mineral resource distributions in continental sedimentary basins(Sun and Wang,2009),and the investigation of climate patterns and ecosystems(Cox,2016).

Exploring the Big Data Using a Rigorous and Quantitative Causality Analysis

Causal analysis is a powerful tool to unravel the data complexity and hence provide clues to achieving, say, better platform design, efficient interoperability and service management, etc. Data science will surely benefit from the advancement in this field. Here we introduce into this community a recent finding in physics on causality and the subsequent rigorous and quantitative causality analysis. The resulting formula is concise in form, involving only the common statistics namely sample covariance. A corollary is that causation implies correlation, but not vice versa, resolving the long-standing philosophical debate over correlation versus causation. The applicability to big data analysis is validated with time series purportedly generated with hidden processes. As a demonstration, a preliminary application to the gross domestic product (GDP) data of United States, China, and Japan reveals some subtle USA-China-Japan relations in certain periods.

Events Sourcing and Command Query Responsibility Segregation Based Fast Data Architecture

With the advent of Big Data, the fields of Statistics and Computer Science coexist in current information systems. In addition to this, technological advances in embedded systems, in particular Internet of Things technologies, make it possible to develop real-time applications. These technological developments are disrupting Software Engineering because the use of large amounts of real-time data requires advanced thinking in terms of software architecture. The purpose of this article is to propose an architecture unifying not only Software Engineering and Big Data activities, but also batch and streaming architectures for the exploitation of massive data. This architecture has the advantage of making possible the development of applications and digital services exploiting very large volumes of data in real time;both for management needs and for analytical purposes. This architecture was tested on COVID-19 data as part of the development of an application for real-time monitoring of the evolution of the pandemic in Côte d’Ivoire using PostgreSQL, ELasticsearch, Kafka, Kafka Connect, NiFi, Spark, Node-Red and MoleculerJS to operationalize the architecture.

全球矿床数据库建设现状、应用与展望

基于大数据和人工智能技术的数据驱动科学范式推动地球科学研究发生了变革。作为地球科学的重要分支,现代矿床学经历了百余年的发展,已经积累了海量的数据资料,这些数据的流通和共享是发挥其资源价值的关键。文章介绍了中国“地质云”与全球矿产资源储量动态评估数据库、澳大利亚深部地球探测计划AuScope、美国矿产资源在线空间数据库、国际经济地质学家学会(SEG)Geofacets数据库、美国标准普尔公司SNL Metals&Mining数据库等国际主要矿床数据库的情况;同时,列举了应用大数据思维和人工智能方法在区域成矿规律、矿床成因机制、矿床类型判别、资源潜力评价、战略咨询等方面取得的若干重要进展。文章提出,未来在深时数字地球国际大科学计划的平台下,整合全球海量矿床数据,建设开放、共享、统一的矿床大数据平台势在必行。

城市中心区轨道交通站点步行接驳路径特征及优化

轨道交通站点步行接驳路径特征研究,对提升轨道交通承担率具有重要意义。基于百度时空大数据API,运用空间分析法、情景模拟法、视觉语义分割法,以天津市营口道站点为例,构建城市中心区轨道交通站点步行接驳路径特征分析方法,挖掘步行接驳路径特征并揭示成因,提出针对性的优化对策。研究表明,目标站点步行接驳效率南部高于北部,具体表现在路段重复率分布均匀度南部大于北部,差异路径分布数量北部多于南部,超过3/4路径的非直线系数偏高;除站点周边区域外,接驳路径品质西北部优于东南部,具体表现在西北部视觉出行环境建设最优,区域整体设施服务水平良好。根据步行接驳路径特征,提出路网结构调整、道路断面优化、出入口调整、绿化品质提升的优化对策。

非结构化高维大数据异常流量时间点挖掘算法

非结构化数据的维度较高,每个样本数据包含的特征非常多,导致了维度灾难问题,使得降低维度并保持有效特征提取难度较大,影响大数据流量异常时间点挖掘的精度。为此,提出新的基于空间映射的非结构化高维大数据流量异常时间点挖掘方法。通过近似解集的几何特征建立稀疏回归模型,求解高维目标空间映射到低维目标子空间的稀疏投影矩阵。根据密度分布选择出一个高密度集合作为聚类中心的候选集,确定聚类的初始聚类中心。同时对聚类形成的各个簇采用剪枝算法,选择时间点候选集,对候选集展开二次判断,挖掘高维大数据流量异常时间点。实验结果表明,数据的降维能有效提高流量异常挖掘精度。相比之下,所提方法的高维大数据流量异常时间点挖掘更加精准,耗时更短。

基于游记大数据的环巢湖地区乡村旅游客源市场时空结构分析

以环巢湖地区作为研究对象,基于2016—2021年旅游网站的乡村旅游游记数据,结合统计分析以及ArcGIS空间分析法研究环巢湖地区乡村旅游客源市场的结构特征以及影响因素。结果表明:乡村旅游客源市场季节性明显,节假日及周末效应突出,其中4至10月为旅游旺季,11至3月为旅游淡季;乡村旅游客源市场的地理集中指数总体较高,客源呈集中分布特征,合肥市为主要客源地,客源吸引半径均值为219.59 km,属于近程客源市场,空间使用曲线是基本型曲线、“U”型曲线和Maxwell-Boltzman曲线3种曲线的复合,客源核心逐渐从长三角地区、京津冀地区向内陆地区、珠三角地区扩散。乡村旅游客源市场的时空结构是多种因素相互作用的结果,主要是由乡村旅游目的地的特殊性决定的,同时闲暇时间、气候舒适度、交通、客源地经济发展水平等也是重要影响因素。

面向周期性工业时序数据的流式清洗系统

为了高效清洗具有时序性、周期性等特点的工业数据,首先利用分布式组件设计了一套流式清洗系统,系统以Mosquitto作为采集数据的汇集中心,以Flume为连接组件,以Kafka为缓冲组件,对接数据清洗组件,使系统具有高吞吐、大缓冲等优势。然后基于速度约束模型,设计了一种周期性数据清洗算法,综合工业数据的时序性、周期性、物理意义等特性,在原有速度约束算法基础上增加周期性检测和数据切片机制,以解决速度约束算法处理周期性数据的失真问题,提高可用度。最后文中以盾构掘进数据集为样本,验证了系统和算法的有效性,以及改进算法的适用性。

基于谱域超图卷积网络的交通流预测模型

针对传统图结构难以对节点间的隐含复杂关联关系建模的问题,利用超图对交通流数据进行高阶表示,提出基于谱域超图卷积网络的交通流预测方法。首先,通过动态超边刻画数据特征层面的关系,利用谱域超图卷积,包括基于傅里叶和图小波的超图卷积及门控时序卷积,在多尺度上提取交通流的时空特征,实现端到端的节点级交通流预测。然后,采用北京市以及美国加利福尼亚州真实历史数据集进行预测实验。消融实验通过孤立和重构网络模型验证了所提方法的有效性。全时段和早高峰交通流预测的实验结果表明,该方法预测准确率高于目前主流交通流预测模型。

基于多突变点与模板匹配的用电设备在线状态监测方法

针对现阶段用电设备状态监测技术存在的处理速度较慢、准确率较低等问题,文中基于多突变点检测和模板匹配策略提出了一种用电设备在线状态监测方法。该方法在缓冲区模型和滑动窗口模型的基础上,利用多路搜索树突变点检测(Ternary Search Tree and Kolmogorov-Smirnov,TSTKS)算法形成窗口维度和缓冲区维度的特征向量,通过两种维度的模板匹配实现用电设备的运行状态匹配和状态切换时刻定位。基于家用电冰箱的仿真实验结果表明,所提方法具有检测速度快、准确率高等优点,可为用电设备状态监测领域提供参考。

一种基于深度学习的时序病变数据段分类方法

针对在大规模时序医疗数据的分析中现有检测方法检测精度低、检测速度慢等问题,文中提出了一种基于深度学习的时序病变数据段分类方法。该方法在TSTKS(Ternary Search Trees and modified Kolmogorov-Smirnov)算法和滑动窗口理论的基础上,利用深度学习技术实现了对病变数据段的快速准确分类。文中以利用该方法对病变数据段进行分类的结果作为依据,实现了滑动窗口大小的动态调整。通过对真实癫痫脑电信号(Electroencephalogram,EEG)进行分析,证明了所提病变数据段分类方法和基于该分类方法的滑动窗口动态调整机制具有检测速度快、精度较高等优点,可以为大规模时序数据的快速分析研究提供一种新选择。

职住双重锚点:中心城区社区公共绿地服务范围的解析——以上海市黄浦区为例

社区公共绿地是社区层级的重要公共基础设施,准确理解其服务范围对规划治理具有现实意义。针对中心城区社区公共绿地的超距离服务现象和就业人口的双生活圈设施使用现象,提出职住双锚点假设。以上海市黄浦区为案例,探索双锚点下访问距离的可解释性、访问时间的聚集表现和居住地的空间分布,并讨论超距离服务范围的影响因素。结果显示,相比传统单居住锚点,职住双锚点分析模式增强了中心城区社区公共绿地服务范围的解释性,中心城区绿地服务范围受职住距离影响,且职住锚点具有不同的访问高峰时段。中心城区社区公共绿地的超远服务距离构成职住分离的侧写,在公共绿地的治理规划工作中应当考虑职住活动的影响。

基于流计算和大数据平台的实时交通流预测

目前交通流预测实时性差,很难满足在线分析和预测任务的需求,基于此提出一种Flink流计算框架和大数据平台结合的实时交通流预测方法。基于流计算框架实时捕捉和预处理数据,包括采用Flink的transform算子对数据进行校验和处理,将处理后的数据sink到大数据的HDFS文件系统,交由下一步的大数据并行框架进行分析建模与训练,实现基于流计算和大数据平台的实时交通流预测。实验结果表明,Flink能够实时捕捉和预处理交通流数据,把数据准时无误送入分布式文件系统中,在此基础上借助大数据框架下的并行分析和建模优势,在实时性数据分析与预测方面取得了较好的效果。

基于大数据技术的皮带预警系统设计

皮带运输机作为矿山重要的运输设备,其性能直接影响生产效率和运行成本。针对现场巡查才能发现皮带运输机异常的问题,本文运用大数据技术对皮带电流进行归纳和分析,设计开发了一套智能化的皮带预警系统,能够及时、准确、有效地发出警报,实现了对皮带异常状态的提前预警。结果显示,预警系统投入使用后,故障平均诊断时间由60 min缩短为5min,减少了系统停机的次数,提高了选矿厂的生产效率。