Research on Optimization of Random Forest Algorithm Based on Spark

As society has developed,increasing amounts of data have been generated by various industries.The random forest algorithm,as a classification algorithm,is widely used because of its superior performance.However,the random forest algorithm uses a simple random sampling feature selection method when generating feature subspaces which cannot distinguish redundant features,thereby affecting its classification accuracy,and resulting in a low data calculation efficiency in the stand-alone mode.In response to the aforementioned problems,related optimization research was conducted with Spark in the present paper.This improved random forest algorithm performs feature extraction according to the calculated feature importance to form a feature subspace.When generating a random forest model,it selects decision trees based on the similarity and classification accuracy of different decision.Experimental results reveal that compared with the original random forest algorithm,the improved algorithm proposed in the present paper exhibited a higher classification accuracy rate and could effectively classify data.

A Dynamic Memory Allocation Optimization Mechanism Based on Spark

Spark is a distributed data processing framework based on memory.Memory allocation is a focus question of Spark research.A good memory allocation scheme can effectively improve the efficiency of task execution and memory resource utilization of the Spark.Aiming at the memory allocation problem in the Spark2.x version,this paper optimizes the memory allocation strategy by analyzing the Spark memory model,the existing cache replacement algorithms and the memory allocation methods,which is on the basis of minimizing the storage area and allocating the execution area according to the demand.It mainly including two parts:cache replacement optimization and memory allocation optimization.Firstly,in the storage area,the cache replacement algorithm is optimized according to the characteristics of RDD Partition,which is combined with PCA dimension.In this section,the four features of RDD Partition are selected.When the RDD cache is replaced,only two most important features are selected by PCA dimension reduction method each time,thereby ensuring the generalization of the cache replacement strategy.Secondly,the memory allocation strategy of the execution area is optimized according to the memory requirement of Task and the memory space of storage area.In this paper,a series of experiments in Spark on Yarn mode are carried out to verify the effectiveness of the optimization algorithm and improve the cluster performance.

A dual-modal probe for NIR fluorogenic and ratiometric photoacoustic imaging of Cys/Hcy in vivo

Biothiols, such as cysteine(Cys) and homocysteine(Hcy), play vital roles in biological homeostasis and are closely related to various pathological and physiological processes in the living systems. Therefore, the in vivo detection of biothiols is of great importance for early diagnosis of diseases and assessment of disease progression. In this work, we developed a near-infrared(NIR) fluorescence and photoacoustic dual-modal molecular probe(NIR-S) that can be specifically activated by Cys or Hcy. The aryl-thioether substituted cyanine probe can undergo nucleophilic substitution and Smiles rearrangement reaction, resulting in specific turn-on NIR fluorescence and ratiometric photoacoustic responses for Hcy/Cys. Thus, NIR-S not only realizes the specific NIR fluorescence and photoacoustic dual mode imaging to detect Hcy/Cys in solution, but also can be applied to living cells and mice to detect Hcy/Cys. This work provided a practical tool to detect Hcy/Cys levels in vivo, which would be beneficial for the early diagnosis and progress of diseases.