Deep Learning-Driven Data Curation and Model Interpretation for Smart Manufacturing

Characterized by self-monitoring and agile adaptation to fast changing dynamics in complex production environments,smart manufacturing as envisioned under Industry 4.0 aims to improve the throughput and reliability of production beyond the state-of-the-art.While the widespread application of deep learning(DL)has opened up new opportunities to accomplish the goal,data quality and model interpretability have continued to present a roadblock for the widespread acceptance of DL for real-world applications.This has motivated research on two fronts:data curation,which aims to provide quality data as input for meaningful DL-based analysis,and model interpretation,which intends to reveal the physical reasoning underlying DL model outputs and promote trust from the users.This paper summarizes several key techniques in data curation where breakthroughs in data denoising,outlier detection,imputation,balancing,and semantic annotation have demonstrated the effectiveness in information extraction from noisy,incomplete,insufficient,and/or unannotated data.Also highlighted are model interpretation methods that address the“black-box”nature of DL towards model transparency.

Earthquake-triggered landslide interpretation model of high resolution remote sensing imageries based on bag of visual word

Traditional visual interpretation is often inefficient due to its excessively workload professional knowledge and strong subjectivity.Therefore,building an automatic interpretation model on high spatial resolution remote sensing images is the key to the quick and efficient interpretation of earthquake-triggered landslides.Aiming at addressing this problem,a landslide interpretation model of high-resolution images based on bag of visual word(BoVW)feature was proposed.The high-resolution images were pre-processed,and then BoVW feature and support vector machine(SVM)was adopted to establish an automatic landslide interpretation model.This model was further compared with the currently widely used Histogram of Oriented Gradient(HoG)feature extraction model.In order to test the effectiveness of the method,typical landslide images were selected to construct a landslide sample library,which was subsequently utilized as the foundation for conducting an experimental study.The results show that the accuracy of landslide extraction using this method reaches as high as 89%,indicating that the method can be used for the automatic interpretation of landslides in disaster-prone areas,and has high practical value for regional disaster prevention and damage reduction.

Application of STEEP and Interpretive Structural Modeling in the Design Imagery of Taiwan Public Ceramic Relief Murals

Ceramic relief mural is a contemporary landscape art that is carefully designed based on human nature,culture,and architectural wall space,combined with social customs,visual sensibility,and art.It may also become the main axis of ceramic art in the future.Taiwan public ceramic relief murals(PCRM)are most distinctive with the PCRM pioneered by Pan-Hsiung Chu of Meinong Kiln in 1987.In addition to breaking through the limitations of traditional public ceramic murals,Chu leveraged local culture and sensibility.The theme of art gives PCRM its unique style and innovative value throughout the Taiwan region.This study mainly analyzes and understands the design image of public ceramic murals,taking Taiwan PCRM’s design and creation as the scope,and applies STEEP analysis,that is,the social,technological,economic,ecological,and political-legal environments are analyzed as core factors;eight main important factors in the artistic design image of ceramic murals are evaluated.Then,interpretive structural modeling(ISM)is used to establish five levels,analyze the four main problems in the main core factor area and the four main target results in the affected factor area;and analyze the problem points and target points as well as their causal relationships.It is expected to sort out the relationship between these factors,obtain the hierarchical relationship of each factor,and provide a reference basis and research methods.

Optimized Non-hyperbolic Stack Imaging Based on Interpretation Model

In complex media, especially for seismic prospecting in deep layers in East China and in the mountainous area in West China, due to the complex geological condition, the common-mid-point （CMP） gather of deep reflection event is neither hyperbolic, nor any simple function. If traditional normal move-out （NMO） and stack imaging technology are still used, it is difficult to get a clear stack image. Based on previous techniques on non-hyperbolic stack, it is thought in this paper that no matter how complex the geological condition is, in order to get an optimized stack image, the stack should be non time move-out stack, and any stacking method limited to some kind of curve will be restricted to application conditions. In order to overcome the above-mentioned limit, a new method called optimized non-hyperbolic stack imaging based on interpretation model is presented in this paper. Based on CMP/CRP （Common-Reflection-Point） gather after NMO or pre-stack migration, this method uses the interpretation model of reflectors as constraint, and takes comparability as a distinguishing criterion, and finally forms a residual move-out correction for the gather of constrained model. Numerical simulation indicates that this method could overcome the non hyperbolic problem and get fine stack image.

Research on Well Testing Interpretation of Low Permeability Deformed Dual Medium Reservoir

Considering the influence of quadratic gradient term and medium deformation on the seepage equation, a well testing interpretation model for low permeability and deformation dual medium reservoirs was derived and established. The difference method was used to solve the problem, and pressure and pressure derivative double logarithmic curves were drawn to analyze the seepage law. The research results indicate that the influence of starting pressure gradient and medium deformation on the pressure characteristic curve is mainly manifested in the middle and late stages. The larger the value, the more obvious the upward warping of the pressure and pressure derivative curve;the parameter characterizing the dual medium is the crossflow coefficient. The channeling coefficient determines the time and location of the appearance of the “concave”. The smaller the value, the later the appearance of the “concave”, and the more to the right of the “concave”.

Analysis of Ecosystem Degradation Factors in Yuanmou Arid-Hot Valleys Based on Interpretative Structural Model

For ecological restoration and reconstruction of the degraded area, it is an important premise to correctly understand the degradation factors of the ecosystem in the arid-hot valleys. The factors including vegetation degradation, land degradation, arid climate, policy failure, forest fire, rapid population growth, excessive deforestation, overgrazing, steep slope reclamation, economic poverty, engineering construction, lithology, slope, low cultural level, geological hazards, biological disaster, soil properties etc, were selected to study the Yuanmou arid-hot valleys. Based on the interpretative structural model （ISM）, it has found out that the degradation factors of the Yuanmou arid-hot valleys were not at the same level but in a multilevel hierarchical system with internal relations, which pointed out that the degradation mode of the arid-hot valleys was ＂straight （appearance）-penetrating-background＂. Such researches have important directive significance for the restoration and reconstruction of the arid-hot valleys ecosystem.

Analysis of Feature Importance and Interpretation for Malware Classification

This study was conducted to enable prompt classification of malware,which was becoming increasingly sophisticated.To do this,we analyzed the important features of malware and the relative importance of selected features according to a learning model to assess how those important features were identified.Initially,the analysis features were extracted using Cuckoo Sandbox,an open-source malware analysis tool,then the features were divided into five categories using the extracted information.The 804 extracted features were reduced by 70%after selecting only the most suitable ones for malware classification using a learning model-based feature selection method called the recursive feature elimination.Next,these important features were analyzed.The level of contribution from each one was assessed by the Random Forest classifier method.The results showed that System call features were mostly allocated.At the end,it was possible to accurately identify the malware type using only 36 to 76 features for each of the four types of malware with the most analysis samples available.These were the Trojan,Adware,Downloader,and Backdoor malware.

Knowledge-Based Multifaceted Modeling Methodology for Open Complex Giant Systems

In this paper, the structure characteristics of open complex giant systems are concretely analysed in depth, thus the view and its significance to support the meta synthesis engineering with manifold knowledge models are clarified. Furthermore, the knowledge based multifaceted modeling methodology for open complex giant systems is emphatically studied. The major points are as follows: (1) nonlinear mechanism and general information partition law; (2) from the symmetry and similarity to the acquisition of construction knowledge; (3) structures for hierarchical and nonhierarchical organizations; (4) the integration of manifold knowledge models; (5) the methodology of knowledge based multifaceted modeling.

Experimental study of gas-water elongated bubble flow during production logging

In order to improve the interpretation of production log data on gas-water elongated bubble （EB） flow in horizontal wells, a multi-phase flow simulation device was set up to conduct a series of measurement experiments using air and tap water as test media, which were measured using a real production logging tool （PLT） string at different deviations and in different mixed flow states. By understanding the characteristics and mechanisms of gas-water EB flow in transparent experimental boreholes during production logging, combined with an analysis of the production log response characteristics and experimental production logging flow pattern maps, a method for flow pattern identification relying on log responses and a drift-flux model were proposed for gas-water EB flow. This model, built upon experimental data of EB flow, reveals physical mechanisms of gas-water EB flow during measurement processing. The coefficients it contains are the specific values under experimental conditions and with the PLT string used in our experiments. These coefficients also reveal the interference with original downhole flow patterns by the PLT string. Due to the representativeness that our simulated flow experiments and PLT string possess, the model coefficients can be applied as empirical values of logging interpretation model parameters directly to real production logging data interpretation, when the measurement circumstances and PLT strings are similar.

ISM-based Correlation Analysis of Development Constraints of Forest Resource-exhausted Cities

Forest resource-exhausted cities have to face with various constraints in the acceleration of its urbanization.This paper analyzed major development constraints of these cities,such as unitary economic structure,weakened forest ecological functions,and geographical barriers,and applied ISM method(Interpretive Structural Modeling) to analyze the correlation among the constraints,and gave suggestions for promoting the development of forest resource-exhausted cities.

Analysis on the Structure of Influencing Factors of Sustainable Supply Chain Implementation of Water Diversion Project

The systematic analysis of the hierarchical relationship among the factors affecting the sustainable supply chain implementation of water diversion projects has theoretical value and practical significance for the sustainable development of large-scale water diversion projects. Through the investigation of relevant literature, books, web pages, materials, and discussions with relevant experts and scholars, a total of 23 factors influencing the sustainable supply chain implementation of water diversion projects were identified. Then using ISM (Interpretative Structural Modeling Method) to analyze the causality of each factor, a multi-level hierarchical structure model was obtained. The results showed that: 1) The surface-level influencing factors of the sustainable supply chain implementation of the water diversion project mainly included 8 factors such as water-saving awareness and water-saving intensity in the diversion area, water quality, water pollution and other disasters, effective incentive mechanisms, etc., and surface-level influencing factors were directly related to the sustainable supply chain implementation of water diversion projects. 2) The indirect influencing factors of the sustainable supply chain of water diversion projects included 12 factors such as the water quality and quantity guarantee rate of the supply chain, the government’s enforcement of laws and regulations, water distribution, ecological compensation, and compensation mechanisms for residents in the water source area. Indirect influencing factor scan acts directly on the direct influencing factors, and intervening in the factors that can be controlled by humans is one of the important ways to improve the sustainable operation of water diversion projects. 3) The fundamental influencing factors for the sustainable supply chain implementation of water diversion projects included three factors: Resettlement policy, government financial support, and sound laws and regulations. Deep influencing factors had multi-channel influence and controllability, and intervening in them was the main means to improve the sustainable operation of water diversion projects.

Organic matter occurrence and pore-forming mechanisms in lacustrine shales in China

The evolution of pore structure in shales is affected by both the thermal evolution of organic matter(OM)and by inorganic diagenesis,resulting in a wide variety of pore structures.This paper examines the OM distribution in lacustrine shales and its influence on pore structure,and describes the process of porosity development.The principal findings are:(i)Three distribution patterns of OM in lacustrine shales are distinguished;laminated continuous distribution,clumped distribution,and stellate scattered distribution.The differences in total organic carbon(TOC)content,free hydrocarbon content(S_(1)),and OM porosity among these distribution patterns are discussed.(ii)Porosity is negatively correlated with TOC and plagioclase content and positively correlated with quartz,dolomite,and clay mineral content.(iii)Pore evolution in lacustrine shales is characterized by a sequence of decreasing-increasing-decreasing porosity,followed by continuously increasing porosity until a relatively stable condition is reached.(iv)A new model for evaluating porosity in lacustrine shales is proposed.Using this model,the organic and inorganic porosity of shales in the Permian Lucaogou Formation are calculated to be 2.5%-5%and 1%-6.3%,respectively,which correlate closely with measured data.These findings may provide a scientific basis and technical support for the sweet spotting in lacustrine shales in China.

Validation and Refinement of Two Interpretable Models for Coronavirus Disease 2019 Prognosis Prediction

Objective:To validate two proposed coronavirus disease 2019(COVID-19)prognosis models,analyze the characteristics of different models,consider the performance of models in predicting different outcomes,and provide new insights into the development and use of artificial intelligence(AI)predictive models in clinical decision-making for COVID-19 and other diseases.Materials and Methods:We compared two proposed prediction models for COVID-19 prognosis that use a decision tree and logistic regression modeling.We evaluated the effectiveness of different model-building strategies using laboratory tests and/or clinical record data,their sensitivity and robustness to the timings of records used and the presence of missing data,and their predictive performance and capabilities in single-site and multicenter settings.Results:The predictive accuracies of the two models after retraining were improved to 93.2% and 93.9%,compared with that of the models directly used,with accuracies of 84.3% and 87.9%,indicating that the prediction models could not be used directly and require retraining based on actual data.In addition,based on the prediction model,new features obtained by model comparison and literature evidence were transferred to integrate the new models with better performance.Conclusions:Comparing the characteristics and differences of datasets used in model training,effective model verification,and a fusion of models is necessary in improving the performance of AI models.

Preventive Control for Power System Transient Security Based on XGBoost and DCOPF with Consideration of Model Interpretability

This paper proposes a new approach for online power system transient security assessment(TSA)and preventive control based on XGBoost and DC optimal power flow(DCOPF).The novelty of this proposal is that it applies the XGBoost and data selection method based on the 1-norm distance in local feature importance evaluation which can provide a certain model interpretability.The method of SMOTE+ENN is adopted for data rebalancing.The contingency-oriented XGBoost model is trained with databases generated by time domain simulations to represent the transient security constraint in the DCOPF model,which has a relatively fast speed of calculation.The transient security constrained generation rescheduling is implemented with the differential evolution algorithm,which is utilized to optimize the rescheduled generation in the preventive control.Feasibility and effectiveness of the proposed approach are demonstrated on an IEEE 39-bus test system and a 500-bus operational model for South Carolina,USA.

COMPLEXITY OF SYSTEM MAINTAINABILITY ANALYSIS BASED ON THE INTERPRETIVE STRUCTURAL MODELING METHODOLOGY： TRANSDISCIPLINARY APPROACH

This paper outlines a diagnostic approach to quantify the maintainability of a Commercial off-the-Shelf （COTS）-based system by analyzing the complexity of the deployment of the system components. Interpretive Structural Modeling （ISM） is used to demonstrate how ISM supports in identifying and understanding interdependencies among COTS components and how they affect the complexity of the maintenance of the COTS Based System （CBS）. Through ISM analysis we have determined which components in the CBS contribute most significantly to the complexity of the system. With the ISM, architects, system integrators, and system maintainers can isolate the COTS products that cause the most complexity, and therefore cause the most effort to maintain, and take precautions to only change those products when necessary or during major maintenance efforts. The analysis also clearly shows the components that can be easily replaced or upgraded with very little impact on the rest of the system.

Toward equation structural modeling:an integration of interpretive structural modeling and structural equation modeling

Interpretive structural modeling(ISM)is an interactive process in which a malformed(bad structured)problem is structured into a comprehensive systematic model.Yet,despite many advantages that ISM provides,this method has some shortcomings,the most important one of which is its reliance on participants’intuition and judgment.This problem undermines the validity of ISM.To solve this problem and further enhance the ISM method,the present study proposes a method called equation structural modeling(ESM),which draws on the capacities of structural equation modeling(SEM).As such,ESM provides a statistically verifiable framework and provides a graphical,hierarchical and intuitive model.

Assessment and Optimization of Explainable Machine Learning Models Applied to Transcriptomic Data

Explainable artificial intelligence aims to interpret how machine learning models make decisions,and many model explainers have been developed in the computer vision field.However,understanding of the applicability of these model explainers to biological data is still lacking.In this study,we comprehensively evaluated multiple explainers by interpreting pre-trained models for predicting tissue types from transcriptomic data and by identifying the top contributing genes from each sample with the greatest impacts on model prediction.To improve the reproducibility and interpretability of results generated by model explainers,we proposed a series of optimization strategies for each explainer on two different model architectures of multilayer perceptron(MLP)and convolutional neural network(CNN).We observed three groups of explainer and model architecture combinations with high reproducibility.Group II,which contains three model explainers on aggregated MLP models,identified top contributing genes in different tissues that exhibited tissue-specific manifestation and were potential cancer biomarkers.In summary,our work provides novel insights and guidance for exploring biological mechanisms using explainable machine learning models.

Algal community structure prediction by machine learning

The algal community structure is vital for aquatic management.However,the complicated environmental and biological processes make modeling challenging.To cope with this difficulty,we investigated using random forests(RF)to predict phytoplankton community shifting based on multi-source environmental factors(including physicochemical,hydrological,and meteorological variables).The RF models robustly predicted the algal communities composed by 13 major classes(Bray-Curtis dissimilarity=9.2±7.0%,validation NRMSE mostly<10%),with accurate simulations to the total biomass(validation R^(2)>0.74)in Norway's largest lake,Lake Mjosa.The importance analysis showed that the hydro-meteorological variables(Standardized MSE and Node Purity mostly>0.5)were the most influential factors in regulating the phytoplankton.Furthermore,an in-depth ecological interpretation uncovered the interactive stress-response effect on the algal community learned by the RF models.The interpretation results disclosed that the environmental drivers(i.e.,temperature,lake inflow,and nutrients)can jointly pose strong influence on the algal community shifts.This study highlighted the power of machine learning in predicting complex algal community structures and provided insights into the model interpretability.

Machine learning-accelerated discovery of novel 2D ferromagnetic materials with strong magnetization

Two-dimensional ferromagnetic(2DFM)semiconductors(metals,half-metals,and so on)are important materials fornext-generation nano-electronic and nano-spintronic devices.However,these kinds of materials remain scarce,“trial anderror”experiments and calculations are both time-consumingand expensive.In the present work,in order to obtain theoptimal 2DFM materials with strong magnetization,a machinelearning(ML)framework was established to search the 2Dmaterial space containing over 2417 samples and identified 615compounds whose magnetic orders were then determined viahigh-throughput first-principles calculations.With the adoptionof ML algorithms,two classification models and a regressionmodel were trained.The interpretability of the regressionmodel was evaluated through Shapley Additive exPlanations(SHAP)analysis.Unexpectedly,it is found that Cr2NF2 is apotential antiferromagnetic ferroelectric 2D multiferroic material.More importantly,60 novel 2DFM candidates werepredicted,and among them,13 candidates have magnetic moments of>7μB.Os2Cl8,Fe3GeSe2,and Mn4N3S2 were predictedto be novel 2DFM semiconductors,metals,and half-metals,respectively.With the adoption of the ML approach in thecurrent work,the prediction of 2DFM materials with strongmagnetization can be accelerated,and the computation timecan be drastically reduced by more than one order ofmagnitude.

数据驱动框架下基于基因表达编程的非线性K-L湍流混合模型

由Rayleigh-Taylor(RT)、Richtmyer-Meshkov(RM)、Kelvin-Helmholtz(KH)等流体力学界面不稳定性诱导的湍流混合现象广泛存在于自然界和工程问题中,准确预测其演化具有十分重要的意义.考虑到实际问题的高雷诺数及复杂性,在可预见的未来,雷诺平均(RANS)方法仍将是工程实践中最具可实现性的选择.一般而言,传统RANS混合模型中至关重要的雷诺应力项是基于经典的Boussinesq线性涡粘假设封闭的.然而,这种线性模型无法充分描述在实际工程流动中发挥着重要作用的湍流各向异性特征.相比之下,非线性模型在这方面具有显著优势.本研究首次将基因表达编程(GEP)方法应用于湍流混合问题,发展了数据驱动的非线性K-L湍流混合模型.与常见的“黑箱”型机器学习模型不同,GEP模型显式给出模型方程,从而具有更强的物理可解释性.具体地,本研究基于二阶截断的广义Cayley-Hamilton非线性本构关系,利用GEP方法中的符号回归功能,形式化地给出了待封闭的模型系数与伽利略不变量之间的函数关系.此外,为了保证训练模型的物理性,我们将可实现性原则引入到了惩罚函数中.模型泛化性测试的结果表明:尽管新模型仅利用倾斜RT混合问题训练得到,但对于几个典型的混合问题均具有良好的鲁棒性.与标准的K-L模型相比,新模型不但具有更高的预测精度,而且能够更好地捕捉湍流的物理特性.此外,通过分析显式的封闭模型,本文进一步给出了新模型的预测效果得以提升的原因.