Big Data Application Simulation Platform Design for Onboard Distributed Processing of LEO Mega-Constellation Networks

Due to the restricted satellite payloads in LEO mega-constellation networks(LMCNs),remote sensing image analysis,online learning and other big data services desirably need onboard distributed processing(OBDP).In existing technologies,the efficiency of big data applications(BDAs)in distributed systems hinges on the stable-state and low-latency links between worker nodes.However,LMCNs with high-dynamic nodes and long-distance links can not provide the above conditions,which makes the performance of OBDP hard to be intuitively measured.To bridge this gap,a multidimensional simulation platform is indispensable that can simulate the network environment of LMCNs and put BDAs in it for performance testing.Using STK's APIs and parallel computing framework,we achieve real-time simulation for thousands of satellite nodes,which are mapped as application nodes through software defined network(SDN)and container technologies.We elaborate the architecture and mechanism of the simulation platform,and take the Starlink and Hadoop as realistic examples for simulations.The results indicate that LMCNs have dynamic end-to-end latency which fluctuates periodically with the constellation movement.Compared to ground data center networks(GDCNs),LMCNs deteriorate the computing and storage job throughput,which can be alleviated by the utilization of erasure codes and data flow scheduling of worker nodes.

The development of data acquisition and processing application system for RF ion source

As the key ion source component of nuclear fusion auxiliary heating devices, the radio frequency （RF） ion source is developed and applied gradually to offer a source plasma with the advantages of ease of control and high reliability. In addition, it easily achieves long-pulse steady-state operation. During the process of the development and testing of the RF ion source, a lot of original experimental data will be generated. Therefore, it is necessary to develop a stable and reliable computer data acquisition and processing application system for realizing the functions of data acquisition, storage, access, and real-time monitoring. In this paper, the development of a data acquisition and processing application system for the RF ion source is presented. The hardware platform is based on the PXI system and the software is programmed on the LabVIEW development environment. The key technologies that are used for the implementation of this software programming mainly include the long-pulse data acquisition technology, multi- threading processing technology, transmission control communication protocol, and the Lempel-Ziv-Oberhumer data compression algorithm. Now, this design has been tested and applied on the RF ion source. The test results show that it can work reliably and steadily. With the help of this design, the stable plasma discharge data of the RF ion source are collected, stored, accessed, and monitored in real-time. It is shown that it has a very practical application significance for the RF experiments.

Optimizing Healthcare Big Data Processing with Containerized PySpark and Parallel Computing: A Study on ETL Pipeline Efficiency

In this study, we delve into the realm of efficient Big Data Engineering and Extract, Transform, Load (ETL) processes within the healthcare sector, leveraging the robust foundation provided by the MIMIC-III Clinical Database. Our investigation entails a comprehensive exploration of various methodologies aimed at enhancing the efficiency of ETL processes, with a primary emphasis on optimizing time and resource utilization. Through meticulous experimentation utilizing a representative dataset, we shed light on the advantages associated with the incorporation of PySpark and Docker containerized applications. Our research illuminates significant advancements in time efficiency, process streamlining, and resource optimization attained through the utilization of PySpark for distributed computing within Big Data Engineering workflows. Additionally, we underscore the strategic integration of Docker containers, delineating their pivotal role in augmenting scalability and reproducibility within the ETL pipeline. This paper encapsulates the pivotal insights gleaned from our experimental journey, accentuating the practical implications and benefits entailed in the adoption of PySpark and Docker. By streamlining Big Data Engineering and ETL processes in the context of clinical big data, our study contributes to the ongoing discourse on optimizing data processing efficiency in healthcare applications. The source code is available on request.

Design of Gamma Spectrum Data Acquisition and Processing System Based on ARM / DSP

Research for detecting or obtaining radionuclide by gamma energy spectrum data acquisition and process system is one of the key issues about intelligent measurement of gamma-ray spectrum. For this reason, a software and hardware implementation schematic design based on ARM ( Advanced RISC Machines) + DSP ( Digital Signal Processor) architecture for gamma energy spectrum data acquisition and processing system is proposed. The paper discusses in detail some key technologies such as communication interface design between microcontroller ARM and digital signal processor DSP,distribution scheduling under multi-task in the ARM-Linux,DSP handling procedures for multi-channel A / D high-speed sample. At the same time,because the traditional Gaussian fitting to determine the boundary of peak is not ideal,it puts forward a weighting factor of Gaussian function least squares fitting realize boundary determined. Finally gamma-spectrum data from sodium iodide NaI( TI) scintillation detector is tested and processed in the new system. The results show that gamma energy spectrum data acquisition and process system is perfect functionality, stable and convergence in unimodal. Compared with data from conventional energy spectrometers,the system can keep better energy resolution in a wide range of pulse pass rate.

A new data acquisition and processing system for profiling sonar

A multi-beam chirp sonar based on IP connections and DSP processing nodes was proposed and designed to provide an expandable system with high-speed processing and mass-storage of real-time signals for multi-beam profiling sonar.The system was designed for seabed petroleum pipeline detection and orientation,and can receive echo signals and process the data in real time,refreshing the display 10 times per second.Every node of the chirp sonar connects with data processing nodes through TCP/IP. Merely by adding nodes,the system’s processing ability can be increased proportionately without changing the software.System debugging and experimental testing proved the system to be practical and stable.This design provides a new method for high speed active sonar.

Analysis on the Applicability of Meter Data Acquisition Methods Based on the Analytic Hierarchy Process

The choice of meter data acquisition methods has important significance for the electric energy management. Based on the comprehensive analysis of several meter data acquisition methods, this paper assess the performance of each one by analytic hierarchy process. We can draw a conclusion by calculating＂ The local automatic meter reading, the prepaid electric energy metering and the remote automatic meter reading have almost the same performance. They are better than the manual meter reading and the vehicle mounted mobile automatic meter reading. So we can choose any one of the three. Among them, the prepaid electric energy metering performs best. This can be a reference for grid company＇ s decision.

An Application of a Multi-Tier Data Warehouse in Oil and Gas Drilling Information Management

Expenditure on wells constitute a significant part of the operational costs for a petroleum enterprise, where most of the cost results from drilling. This has prompted drilling departments to continuously look for ways to reduce their drilling costs and be as efficient as possible. A system called the Drilling Comprehensive Information Management and Application System (DCIMAS) is developed and presented here, with an aim at collecting, storing and making full use of the valuable well data and information relating to all drilling activities and operations. The DCIMAS comprises three main parts, including a data collection and transmission system, a data warehouse (DW) management system, and an integrated platform of core applications. With the support of the application platform, the DW management system is introduced, whereby the operation data are captured at well sites and transmitted electronically to a data warehouse via transmission equipment and ETL (extract, transformation and load) tools. With the high quality of the data guaranteed, our central task is to make the best use of the operation data and information for drilling analysis and to provide further information to guide later production stages. Applications have been developed and integrated on a uniform platform to interface directly with different layers of the multi-tier DW. Now, engineers in every department spend less time on data handling and more time on applying technology in their real work with the system.

A Review of Data Acquisition System based on LabVIEW

With the continuous development and expansion of automotive industry, car safety problems are widely concerned by most of us. In order to make it easier to adjust car′s real-time body posture, it is necessary to design a real-time data acquisition system, thus reducing the number of casualties in normal driving. A brief account of the development status of data acquisition system at home and abroad, and the concept, principle,function of virtual instrument platform are given. The advantages and disadvantages of the data acquisitionsystem, based on LabVIEW in different methods of data acquisition, are analyzed and compared, and put forward some improvements and optimization, so the real-time data acquisition system can meet the requirement of adjusting the state of the vehicle, improving real-time of data acquisition and processing. Finally, it indicates the development direction and prospect of data acquisition system based on LabVIEW.

Principle and realization method of tunnel deformation detection based on image recognition and data transmission technology

To detect the deformation of the tunnel structure based on image sensor networks is the advanced study and application of spatial sensor technology. For the vertical settlement of metro tunnel caused by internal and external stress after its long period operation, the overall scheme and measuring principle of tunnel deformation detection system is in- troduced. The image data acquisition and processing of detection target are achieved by the cooperative work of image sensor, ARM embedded system. RS485 communication achieves the data transmission between ARM memory and host computer. The database system in station platform analyses the detection data and obtains the deformation state of tunnel inner wall, which makes it possible to early-warn the tunnel deformation and take preventive measures in time.

PI DataLink在离子膜工艺生产中的应用

介绍PI DataLink的开发背景。在氯碱企业电解离子膜生产管理过程中,利用PI DataLink软件的强大功能,自主开发离子膜生产监控与异常查询、数据处理的综合管理软件,实现了关键设备数一键监控、消耗一键统计、指标一键管理、报表一键生成。通过该系统,加大了对生产过程和关键设备参数的监控力度,圆满完成了对工艺ABC类指标、经济消耗指标的控制,减少了原料及公用工程的消耗,实现了对工艺过程的在线分析,避免了危险事故的再次发生。后期还要完成全部系统的数据监控以及精确的数据变化趋势。

Data Logging and Control of a Remote Inverter Using LoRa and Power Line Communication

For decades, the power system was highly centralized. With the growing integration of distributed generations into the system, there is a necessity for bi-directional communication methods to monitor and control the remote assets. The primary objective of this paper is to develop a communication link for monitoring and controlling a grid-connected inverter in a remote location. Furthermore, the paper presents developments that have been incorporated to improve the communication link. The literature survey indicates that LoRa is superior compared to other technologies, but has some security and reliability issues. This paper also presents an encryption algorithm to improve the security of the LoRa link. Local data storage added to the system before transmitting data increases the system reliability. A display at the transmission end is added to improve the user-friendliness of the communication link. A Powerline Communication link is parallelly added to the LoRa link to improve the reliability. Finally, tests are conducted with an actual inverter and the results are presented. The tests show that the developed communication link has improved security and reliability, while its open nature makes it highly scalable and adaptable for employment in other smart grid applications.

The Organizing of Database from Surlari National Geomagnetic Observatory

Our paper describes the organizing of database,remarks about SNGO(Surlari National Geomagnetic Observatory)and network infrastructure.Based on the geomagnetic data acquired and stored on the database server,we perform the processing and analysis of geomagnetic parameters through different spectral,statistical and correlation methods.All these parameters are included in the geomagnetic database on server.The web interface for the database meets the different needs of handling the data collected,raw or processed.The server-side programming language used for design is php.This allow us to select different periods for which access to stored data,required for different search filters and different parameters or data from different time periods can be compared.For a more in-depth analysis of the stored data,through JavaScript programming language graphs for different parameters can be drawn.Access to the web interface can be done with or without authentication,depending on the need to ensure the security of certain data collected,stored and processed.The applications are scalable for different devices that will access it:mobile,tablets,laptops or desktops.

Application of FPGA in Process Tomography Systems

This paper will provide some insights on the application of Field Programmable Gate Array (FPGA) in process tomography. The focus of this paper will be to investigate the performance of the technology with respect to various tomography systems and comparison to other similar technologies including the Application Specific Integrated Circuit (ASIC), Graphics Processing Unit (GPU) and the microcontroller. Fundamentally, the FPGA is primarily used in the Data Acquisition System (DAQ) due to its better performance and better trade-off as compared to competitor technologies. However, the drawback of using FPGA is that it is relatively more expensive.

NUMERICAL CALCULATION AND APPLICATION OF CNOIDAL WAVE THEORY

In this paper expressions for the solution and general properties (wave energy, radiation stress, etc.) of the first-order cnoidal wave theory are given as a power series of the complementary modulus of the theta function. These expressions are convenient for numerical calculation because they converge very rapidly over the applicable range of Ursell number (Us> 10) for the theory. As examples of practical application, the relative height of wave crest, maximum bottom velocity, radiation stress, wave set-down and set-up are calculated. Comparisons between calculated values and experimental data are made.

FY-1 METEOROLOGICAL SATELLITE DATA RECEIVING, PROCESSING AND APPLICATION

This paper summarizes the characteristics of Fengyun satellite and the properties of the data transmission, and also gives a brief introduction to the data acquisition and processing system. At last, it describes the products derived from satellite data produced by this system and their applications in particular.

Extending JASAG with data processing techniques for speeding up agricultural simulation applications:A case study with Simugan

Resource-intensive agricultural simulation applications have increased the need for gridification tools–i.e.,software to transform and scale up the applications using Grid infrastructures–.Previous research has proposed JASAG,a generic gridification tool for agricultural applications,through which the performance of a whole-farm simulation application called Simugan improved considerably.However,JASAG still lacks proper support for efficiently exploiting Grid storage resources,causing significant delays for assembling and summarizing the generated data.In this application note,two different data processing techniques in the context of JASAG are presented to tackle this problem.Simugan was again employed to validate the benefits of these techniques.Experiments using data processing techniques show that the execution time of Simugan was accelerated by a factor of up to 34.34.

遥感技术新进展赋能水利行业应用

随着遥感技术的不断发展和水利信息化进程的加快,水利遥感有了飞速的发展,与水利现代化发展同步。目前,遥感技术在水利行业的应用范围非常广,在为水利高质量发展提供有力支撑和强力驱动的数字孪生水利建设中发挥了重要的作用。近十多年来,快速发展的遥感技术为提升水利遥感应用能力提供了强有力的支撑,具体表现在数据源的丰富化和多源化、数据获取的速度和效率显著提升、定量估算与反演精度大大提高、专题信息可获得性不断增加、数据处理和信息提取的智能化水平取得长足进步。介绍了遥感技术发展赋能水利遥感应用领域的现状和前景展望,为遥感技术赋能水利新质生产力提供参考。

基于STM32单片机的二维码门禁控制系统

该文介绍一种基于STM32单片机的二维码门禁控制系统。该系统通过集成图像采集、二维码识别、数据通信和控制逻辑等关键模块,实现高效、安全的门禁管理。首先,系统利用GM65图像传感器捕捉进出人员的二维码图像,然后通过STM32单片机的强大处理能力进行二维码解码,从而获取其中的身份信息。接着,系统将解码得到的数据与One-Net云端信息进行比对,以判断进出人员是否具有合法通行权限。若权限验证通过,则门禁设备执行开锁操作,允许人员通行;若验证不通过,则系统拒绝开锁,并记录相关事件。该系统的引入,不仅提升门禁管理的效率和安全性,还降低了人为操作的错误率,为智能化、自动化的门禁管理提供新的解决方案。

利用PI-Process Book系统实现锅炉安全监控

利用PI-Process Book实时数据采集系统与Excel进行对接,通过收集数据,对锅炉监控数据与正常值进行比对,从而判定锅炉参数、设备运行是否正常,实现对锅炉实时监控,实现机组连续自动运行、全自动报警以及报警情况记录,为锅炉运行提供设备、参数等异常情况的预报警,为机组事故处理提供更充足的时间。