An Innovative K-Anonymity Privacy-Preserving Algorithm to Improve Data Availability in the Context of Big Data

The development of technologies such as big data and blockchain has brought convenience to life,but at the same time,privacy and security issues are becoming more and more prominent.The K-anonymity algorithm is an effective and low computational complexity privacy-preserving algorithm that can safeguard users’privacy by anonymizing big data.However,the algorithm currently suffers from the problem of focusing only on improving user privacy while ignoring data availability.In addition,ignoring the impact of quasi-identified attributes on sensitive attributes causes the usability of the processed data on statistical analysis to be reduced.Based on this,we propose a new K-anonymity algorithm to solve the privacy security problem in the context of big data,while guaranteeing improved data usability.Specifically,we construct a new information loss function based on the information quantity theory.Considering that different quasi-identification attributes have different impacts on sensitive attributes,we set weights for each quasi-identification attribute when designing the information loss function.In addition,to reduce information loss,we improve K-anonymity in two ways.First,we make the loss of information smaller than in the original table while guaranteeing privacy based on common artificial intelligence algorithms,i.e.,greedy algorithm and 2-means clustering algorithm.In addition,we improve the 2-means clustering algorithm by designing a mean-center method to select the initial center of mass.Meanwhile,we design the K-anonymity algorithm of this scheme based on the constructed information loss function,the improved 2-means clustering algorithm,and the greedy algorithm,which reduces the information loss.Finally,we experimentally demonstrate the effectiveness of the algorithm in improving the effect of 2-means clustering and reducing information loss.

The Influence of Social Media on College Choice among Undergraduates Majoring in Big Data Management and Application in China:Taking Guilin University of Electronic Technology as an Example

This study aims to investigate the influence of social media on college choice among undergraduates majoring in Big Data Management and Application in China.The study attempts to reveal how information on social media platforms such as Weibo,WeChat,and Zhihu influences the cognition and choice process of prospective students.By employing an online quantitative survey questionnaire,data were collected from the 2022 and 2023 classes of new students majoring in Big Data Management and Application at Guilin University of Electronic Technology.The aim was to evaluate the role of social media in their college choice process and understand the features and information that most attract prospective students.Social media has become a key factor influencing the college choice decision-making of undergraduates majoring in Big Data Management and Application in China.Students tend to obtain school information through social media platforms and use this information as an important reference in their decision-making process.Higher education institutions should strengthen their social media information dissemination,providing accurate,timely,and attractive information.It is also necessary to ensure effective management of social media platforms,maintain a positive reputation for the school on social media,and increase the interest and trust of prospective students.Simultaneously,educational decision-makers should consider incorporating social media analysis into their recruitment strategies to better attract new student enrollment.This study provides a new perspective for understanding higher education choice behavior in the digital age,particularly by revealing the importance of social media in the educational decision-making process.This has important practical and theoretical implications for higher education institutions,policymakers,and social media platform operators.

Research on a Fog Computing Architecture and BP Algorithm Application for Medical Big Data

Although the Internet of Things has been widely applied,the problems of cloud computing in the application of digital smart medical Big Data collection,processing,analysis,and storage remain,especially the low efficiency of medical diagnosis.And with the wide application of the Internet of Things and Big Data in the medical field,medical Big Data is increasing in geometric magnitude resulting in cloud service overload,insufficient storage,communication delay,and network congestion.In order to solve these medical and network problems,a medical big-data-oriented fog computing architec-ture and BP algorithm application are proposed,and its structural advantages and characteristics are studied.This architecture enables the medical Big Data generated by medical edge devices and the existing data in the cloud service center to calculate,compare and analyze the fog node through the Internet of Things.The diagnosis results are designed to reduce the business processing delay and improve the diagnosis effect.Considering the weak computing of each edge device,the artificial intelligence BP neural network algorithm is used in the core computing model of the medical diagnosis system to improve the system computing power,enhance the medical intelligence-aided decision-making,and improve the clinical diagnosis and treatment efficiency.In the application process,combined with the characteristics of medical Big Data technology,through fog architecture design and Big Data technology integration,we could research the processing and analysis of heterogeneous data of the medical diagnosis system in the context of the Internet of Things.The results are promising:The medical platform network is smooth,the data storage space is sufficient,the data processing and analysis speed is fast,the diagnosis effect is remarkable,and it is a good assistant to doctors’treatment effect.It not only effectively solves the problem of low clinical diagnosis,treatment efficiency and quality,but also reduces the waiting time of patients,effectively solves the contradiction between doctors and patients,and improves the medical service quality and management level.

Metaheuristic Based Clustering with Deep Learning Model for Big Data Classification

Recently,a massive quantity of data is being produced from a distinct number of sources and the size of the daily created on the Internet has crossed two Exabytes.At the same time,clustering is one of the efficient techniques for mining big data to extract the useful and hidden patterns that exist in it.Density-based clustering techniques have gained significant attention owing to the fact that it helps to effectively recognize complex patterns in spatial dataset.Big data clustering is a trivial process owing to the increasing quantity of data which can be solved by the use of Map Reduce tool.With this motivation,this paper presents an efficient Map Reduce based hybrid density based clustering and classification algorithm for big data analytics(MR-HDBCC).The proposed MR-HDBCC technique is executed on Map Reduce tool for handling the big data.In addition,the MR-HDBCC technique involves three distinct processes namely pre-processing,clustering,and classification.The proposed model utilizes the Density-Based Spatial Clustering of Applications with Noise(DBSCAN)techni-que which is capable of detecting random shapes and diverse clusters with noisy data.For improving the performance of the DBSCAN technique,a hybrid model using cockroach swarm optimization(CSO)algorithm is developed for the exploration of the search space and determine the optimal parameters for density based clustering.Finally,bidirectional gated recurrent neural network(BGRNN)is employed for the classification of big data.The experimental validation of the proposed MR-HDBCC technique takes place using the benchmark dataset and the simulation outcomes demonstrate the promising performance of the proposed model interms of different measures.

Research and Application on Spark Clustering Algorithm in Campus Big Data Analysis

Big data analysis has penetrated into all fields of society and has brought about profound changes.However,there is relatively little research on big data supporting student management regarding college and university’s big data.Taking the student card information as the research sample,using spark big data mining technology and K-Means clustering algorithm,taking scholarship evaluation as an example,the big data is analyzed.Data includes analysis of students’daily behavior from multiple dimensions,and it can prevent the unreasonable scholarship evaluation caused by unfair factors such as plagiarism,votes of teachers and students,etc.At the same time,students’absenteeism,physical health and psychological status in advance can be predicted,which makes student management work more active,accurate and effective.

Distributed C-Means Algorithm for Big Data Image Segmentation on a Massively Parallel and Distributed Virtual Machine Based on Cooperative Mobile Agents

The aim of this paper is to present a distributed algorithm for big data classification, and its application for Magnetic Resonance Images (MRI) segmentation. We choose the well-known classification method which is the c-means method. The proposed method is introduced in order to perform a cognitive program which is assigned to be implemented on a parallel and distributed machine based on mobile agents. The main idea of the proposed algorithm is to execute the c-means classification procedure by the Mobile Classification Agents (Team Workers) on different nodes on their data at the same time and provide the results to their Mobile Host Agent (Team Leader) which computes the global results and orchestrates the classification until the convergence condition is achieved and the output segmented images will be provided from the Mobile Classification Agents. The data in our case are the big data MRI image of size (m × n) which is splitted into (m × n) elementary images one per mobile classification agent to perform the classification procedure. The experimental results show that the use of the distributed architecture improves significantly the big data segmentation efficiency.

Unstructured Big Data Threat Intelligence Parallel Mining Algorithm

To efficiently mine threat intelligence from the vast array of open-source cybersecurity analysis reports on the web,we have developed the Parallel Deep Forest-based Multi-Label Classification(PDFMLC)algorithm.Initially,open-source cybersecurity analysis reports are collected and converted into a standardized text format.Subsequently,five tactics category labels are annotated,creating a multi-label dataset for tactics classification.Addressing the limitations of low execution efficiency and scalability in the sequential deep forest algorithm,our PDFMLC algorithm employs broadcast variables and the Lempel-Ziv-Welch(LZW)algorithm,significantly enhancing its acceleration ratio.Furthermore,our proposed PDFMLC algorithm incorporates label mutual information from the established dataset as input features.This captures latent label associations,significantly improving classification accuracy.Finally,we present the PDFMLC-based Threat Intelligence Mining(PDFMLC-TIM)method.Experimental results demonstrate that the PDFMLC algorithm exhibits exceptional node scalability and execution efficiency.Simultaneously,the PDFMLC-TIM method proficiently conducts text classification on cybersecurity analysis reports,extracting tactics entities to construct comprehensive threat intelligence.As a result,successfully formatted STIX2.1 threat intelligence is established.

Study on the Prediction System of Shrimp Field Distribution in the East China Sea Based on Big Data Analysis of Fishing Trajectories

In recent years,China has paid more and more attention to the development of marine economy and the management and protection of fishery resources.The management departments at all levels regulate and manage the fishing behavior of fishing vessels through the data of fishing trajectories.In this paper,the distribution of shrimp farms in the East China Sea is predicted by studying the trajectories and behavior patterns of shrimp boats in the system of fishing trajectories.At the same time,a set of shrimp farm distribution management system based on Back Propagation algorithm is established.It can monitor the trajectories of fishing boats and the distribution of shrimp groups in real time,which effectively improves the work efficiency and management mode of the management department.It also plays a positive role in regulating the behavior of fishing boats at sea.

Manufacturers’channel selections under the influence of the platform with big data analytics

To obtain the platform s big data analytics support,manufacturers in the traditional retail channel must decide whether to use the direct online channel.A retail supply chain model and a direct online supply chain model are built,in which manufacturers design products alone in the retail channel,while the platform and manufacturer complete the product design in the direct online channel.These two models are analyzed using the game theoretical model and numerical simulation.The findings indicate that if the manufacturers design capabilities are not very high and the commission rate is not very low,the manufacturers will choose the direct online channel if the platform s technical efforts are within an interval.When the platform s technical efforts are exogenous,they positively influence the manufacturers decisions;however,in the endogenous case,the platform s effect on the manufacturers is reflected in the interaction of the commission rate and cost efficiency.The manufacturers and the platform should make synthetic effort decisions based on the manufacturer s development capabilities,the intensity of market competition,and the cost efficiency of the platform.

An Approach to Find the Point of Buying Stock Based on Big Data

It is a research subject that has attracted a wide concern and study for a long time to find a suitable trading point of stock.From the views of big data and quantization technique,the paper tries to propose an approach,through the form of algorithm,based on big data analysis and linear weighted moving average curve,to find the point of buying stock,so that the trader would like to achieve the expected profit with a higher probability;and makes the digital experiment to further explain the approach and verify its performance.This work can promote the development of big data research and quantization technique,and can also provide a certain reference method for the trader making the technology analysis of the trade.

A Survey of Machine Learning for Big Data Processing

Today’s world is a data-driven one,with data being produced in vast amounts as a result of the rapid growth of technology that permeates every aspect of our lives.New data processing techniques must be developed and refined over time to gain meaningful insights from this vast continuous volume of produced data in various forms.Machine learning technologies provide promising solutions and potential methods for processing large quantities of data and gaining value from it.This study conducts a literature review on the application of machine learning techniques in big data processing.It provides a general overview of machine learning algorithms and techniques,a brief introduction to big data,and a discussion of related works that have used machine learning techniques in a variety of sectors to process big amounts of data.The study also discusses the challenges and issues associated with the usage of machine learning for big data.

Protection of Basic Human Rights in the Application of Big Data to Counter Terrorism

In the era of big data,the ways people work,live and think have changed dramatically,and the social governance system is also being restructured.Achieving intelligent social governance has now become a national strategy.The application of big data technology to counterterrorism efforts has become a powerful weapon for all countries.However,due to the uncertainty,difficulty of interpretation and potential risk of discrimination in big data technology and algorithm models,basic human rights,freedom and even ethics are likely to be impacted and challenged.As a result,there is an urgent need to prioritize basic human rights and regulate the application of big data for counter terrorism purposes.The legislation and law enforcement regarding the use of big data to counter terrorism must be subject to constitutional and other legal reviews,so as to strike a balance between safeguarding national security and protecting basic human rights.

Energy Theft Detection in Smart Grids with Genetic Algorithm-Based Feature Selection

As big data,its technologies,and application continue to advance,the Smart Grid(SG)has become one of the most successful pervasive and fixed computing platforms that efficiently uses a data-driven approach and employs efficient information and communication technology(ICT)and cloud computing.As a result of the complicated architecture of cloud computing,the distinctive working of advanced metering infrastructures(AMI),and the use of sensitive data,it has become challenging tomake the SG secure.Faults of the SG are categorized into two main categories,Technical Losses(TLs)and Non-Technical Losses(NTLs).Hardware failure,communication issues,ohmic losses,and energy burnout during transmission and propagation of energy are TLs.NTL’s are human-induced errors for malicious purposes such as attacking sensitive data and electricity theft,along with tampering with AMI for bill reduction by fraudulent customers.This research proposes a data-driven methodology based on principles of computational intelligence as well as big data analysis to identify fraudulent customers based on their load profile.In our proposed methodology,a hybrid Genetic Algorithm and Support Vector Machine(GA-SVM)model has been used to extract the relevant subset of feature data from a large and unsupervised public smart grid project dataset in London,UK,for theft detection.A subset of 26 out of 71 features is obtained with a classification accuracy of 96.6%,compared to studies conducted on small and limited datasets.

Algorithms for Pre-Compiling Programs by Parallel Compilers

The paper addresses the challenge of transmitting a big number offiles stored in a data center(DC),encrypting them by compilers,and sending them through a network at an acceptable time.Face to the big number offiles,only one compiler may not be sufficient to encrypt data in an acceptable time.In this paper,we consider the problem of several compilers and the objective is tofind an algorithm that can give an efficient schedule for the givenfiles to be compiled by the compilers.The main objective of the work is to minimize the gap in the total size of assignedfiles between compilers.This minimization ensures the fair distribution offiles to different compilers.This problem is considered to be a very hard problem.This paper presents two research axes.Thefirst axis is related to architecture.We propose a novel pre-compiler architecture in this context.The second axis is algorithmic development.We develop six algorithms to solve the problem,in this context.These algorithms are based on the dispatching rules method,decomposition method,and an iterative approach.These algorithms give approximate solutions for the studied problem.An experimental result is imple-mented to show the performance of algorithms.Several indicators are used to measure the performance of the proposed algorithms.In addition,five classes are proposed to test the algorithms with a total of 2350 instances.A comparison between the proposed algorithms is presented in different tables discussed to show the performance of each algorithm.The result showed that the best algorithm is the Iterative-mixed Smallest-Longest-Heuristic(ISL)with a percentage equal to 97.7%and an average running time equal to 0.148 s.All other algorithms did not exceed 22%as a percentage.The best algorithm excluding ISL is Iterative-mixed Longest-Smallest Heuristic(ILS)with a percentage equal to 21,4%and an average running time equal to 0.150 s.

Blockchain Based Consensus Algorithm and Trustworthy Evaluation of Authenticated Subgraph Queries

Over the past era,subgraph mining from a large collection of graph database is a crucial problem.In addition,scalability is another big problem due to insufficient storage.There are several security challenges associated with subgraph mining in today’s on-demand system.To address this downside,our proposed work introduces a Blockchain-based Consensus algorithm for Authenticated query search in the Large-Scale Dynamic Graphs(BCCA-LSDG).The two-fold process is handled in the proposed BCCA-LSDG:graph indexing and authenticated query search(query processing).A blockchain-based reputation system is meant to maintain the trust blockchain and cloud server of the proposed architecture.To resolve the issues and provide safe big data transmission,the proposed technique also combines blockchain with a consensus algorithm architecture.Security of the big data is ensured by dividing the BC network into distinct networks,each with a restricted number of allowed entities,data kept in the cloud gate server,and data analysis in the blockchain.The consensus algorithm is crucial for maintaining the speed,performance and security of the blockchain.Then Dual Similarity based MapReduce helps in mapping and reducing the relevant subgraphs with the use of optimal feature sets.Finally,the graph index refinement process is undertaken to improve the query results.Concerning query error,fuzzy logic is used to refine the index of the graph dynamically.The proposed technique outperforms advanced methodologies in both blockchain and non-blockchain systems,and the combination of blockchain and subgraph provides a secure communication platform,according to the findings.

协同过滤下混合大数据无损挖掘算法研究

大数据具有大规模性、多样性以及价值性,由于海量数据间的较高相似度,导致数据挖掘过程易受冗余干扰,出现数据丢失、损坏等问题。为解决上述问题,提出基于协同过滤算法的混合大数据无损挖掘方法。对混合大数据集成预处理,去除冗余,将不同来源的相同数据无损融合。采用协同过滤算法的时间衰减函数,计算挖掘项目间相似性。在混合大数据特征关联度的约束下,实现混合大数据无损挖掘。实验结果表明,所提方法应用下,混合大数据量高达25000MB时,数据挖掘所需时间仅为45ms左右,且挖掘精度高达95%以上,数据挖掘结果与目标具有一致性。

Focusing on individualized nutrition within the algorithmic diet: an in-depth look at recent advances in nutritional science, microbial diversity studies, and human health

A healthy balanced diet and a healthy lifestyle are very closely linked.Whichever the biological link is,it is overwhelming to understand.Modifications in how food is served,divided up,and supervised,such as the introduction of nutritional hygiene standards,food handling practices,and the entry of macro and micronutrients,have had a big impact on human health in the last few decades.Growing evidence indicates that our gut microbiota may affect our health in ways that are at least in part influenced by our diet and the ingredients used in the preparation of our food and drinks,as well as other factors.As a new problem,this one is getting a lot of attention,but it would be hard to figure out how the gut microbiota and nutrition molecules work together and how they work in certain situations.Genetic analysis,metagenomic characterization,configuration analysis of foodstuffs,and the shift to digital health information have provided massive amounts of data that might be useful in tackling this problem.Machine learning and deep learning methods will be employed extensively as part of this research in order to blend complicated data frames and extract crucial information that will be capable of exposing and grasping the incredibly delicate links that prevail between diet,gut microbiome,and overall wellbeing.Nutrition,well-being,and gut microorganisms are a few subjects covered in this field.It takes into account not only databases and high-speed technology,but also virtual machine problem-solving skills,intangible assets,and laws.This is how it works:Computer vision,data mining,and analytics are all discussed extensively in this study piece.We also point out limitations in existing methodologies and new situations that discovered in the context of current scientific knowledge in the decades to come.We also provide background on"bioinformatics"algorithms;recent developments may seem to herald a revolution in clinical research,pushing traditional techniques to the sidelines.Furthermore,their true potential rests in their ability to work in conjunction with,rather than as a substitute for,traditional research hypotheses and procedures.When new metadata propositions are made by focusing on easily understandable frameworks,they will always need to be rigorously validated and brought into question.Because of the huge datasets available,assumption analysis may be used to complement rather than a substitute for more conventional concept-driven scientific investigation.It is only by employing all of us that we will all increase the quality of evidence-based practice.

智媒时代互联网广告的算法“黑箱”与治理路径探析

随着互联网技术的更新迭代,人工智能、算法等技术逐渐嵌入日常随处可见的广告中,也因其“黑箱”问题给广告治理带来权力滥用、技术异化、隐私窥视等难题。算法“黑箱”的封闭性、歧视性和风险性等特征使算法在应用中直接影响互联网广告的制度。破解这些难题的路径在于:规制算法权力、改革算法生态、设立数据收集壁垒、推进“黑箱”透明。

增强还是替代:人工智能与人类智能关系的哲学反思

人工智能哲学的核心论争是考察人工智能与人类智能的紧张关系与有机张力。人工智能是一个比照与关联了合成智能、计算智能与增强智能的概念,其建构目标是智能主体,底层逻辑是大数据,运行关键是算法。人工智能的发展形态蕴含着从弱人工智能到强人工智能或通用人工智能的可能路向,甚至不排除在未来演化出超级智能的形态。人工智能与人类智能之间存在着内在的紧张关系:人工智能或以温和的方式实现对人类智能的模拟与增强,或以激进的方式去超越与替代人类智能。人类未来既需要认真对待与重视人工智能,也需要为其构建平等合理的伦理原则。

基于大数据流水线系统的算法模型整合方法研究——以基于机器学习方法的LiDAR数据树木生物量反演为例

【背景】激光雷达(LiDAR)数据在森林资源分析利用方面有着广泛应用,科研人员研制了很多涉及大数据管理和人工智能的专业算法模型,这些算法模型目前多数散落在研究人员手里,尚缺乏新型信息化平台对其进行整合。【方法】大数据流水线系统πFlow软件具有大数据管理能力和大数据算法集成能力,并可以所见即所得方式构建流水线并调度运行流水线,适合于LiDAR数据复杂算法模型的整合,且流水线可定制、可复用。【内容】本文介绍了πFlow的特点和功能,并以基于LiDAR冠层高度模型(CHM)数据的树冠解析及利用机器学习方法估测树木生物量为例,介绍了将算法整合到πFlow并构建LiDAR数据分析处理流水线的方法和技术,且对流水线进行了测试运行。【结果】利用πFlow构建的可重复信息化平台可支撑野外站观测网络的LiDAR数据生物量快速反演,为数据密集型的专业数据处理算法模型的整合提供了创新方法技术。