Tuning-up Learning Parameters for Deep Convolutional Neural Network:A Case Study for Hand-Drawn Sketch Images

Several recent successes in deep learning(DL),such as state-of-the-art performance on several image classification benchmarks,have been achieved through the improved configuration.Hyperparameters(HPs)tuning is a key factor affecting the performance of machine learning(ML)algorithms.Various state-of-the-art DL models use different HPs in different ways for classification tasks on different datasets.This manuscript provides a brief overview of learning parameters and configuration techniques to show the benefits of using a large-scale handdrawn sketch dataset for classification problems.We analyzed the impact of different learning parameters and toplayer configurations with batch normalization(BN)and dropouts on the performance of the pre-trained visual geometry group 19(VGG-19).The analyzed learning parameters include different learning rates and momentum values of two different optimizers,such as stochastic gradient descent(SGD)and Adam.Our analysis demonstrates that using the SGD optimizer and learning parameters,such as small learning rates with high values of momentum,along with both BN and dropouts in top layers,has a good impact on the sketch image classification accuracy.

Learning Bayesian network parameters under new monotonic constraints

When the training data are insufficient, especially when only a small sample size of data is available, domain knowledge will be taken into the process of learning parameters to improve the performance of the Bayesian networks. In this paper, a new monotonic constraint model is proposed to represent a type of common domain knowledge. And then, the monotonic constraint estimation algorithm is proposed to learn the parameters with the monotonic constraint model. In order to demonstrate the superiority of the proposed algorithm, series of experiments are carried out. The experiment results show that the proposed algorithm is able to obtain more accurate parameters compared to some existing algorithms while the complexity is not the highest.

Biomedical Osteosarcoma Image Classification Using Elephant Herd Optimization and Deep Learning

Osteosarcoma is a type of malignant bone tumor that is reported across the globe.Recent advancements in Machine Learning(ML)and Deep Learning(DL)models enable the detection and classification of malignancies in biomedical images.In this regard,the current study introduces a new Biomedical Osteosarcoma Image Classification using Elephant Herd Optimization and Deep Transfer Learning(BOIC-EHODTL)model.The presented BOIC-EHODTL model examines the biomedical images to diagnose distinct kinds of osteosarcoma.At the initial stage,Gabor Filter(GF)is applied as a pre-processing technique to get rid of the noise from images.In addition,Adam optimizer with MixNet model is also employed as a feature extraction technique to generate feature vectors.Then,EHOalgorithm is utilized along with Adaptive Neuro-Fuzzy Classifier(ANFC)model for recognition and categorization of osteosarcoma.EHO algorithm is utilized to fine-tune the parameters involved in ANFC model which in turn helps in accomplishing improved classification results.The design of EHO with ANFC model for classification of osteosarcoma is the novelty of current study.In order to demonstrate the improved performance of BOIC-EHODTL model,a comprehensive comparison was conducted between the proposed and existing models upon benchmark dataset and the results confirmed the better performance of BOIC-EHODTL model over recent methodologies.

Linkage intensity learning approach with genetic algorithm for causality diagram

The causality diagram theory, which adopts graphical expression of knowledge and direct intensity of causality, overcomes some shortages in belief network and has evolved into a mixed causality diagram methodology for discrete and continuous variable. But to give linkage intensity of causality diagram is difficult, particularly in many working conditions in which sampling data are limited or noisy. The classic learning algorithm is hard to be adopted. We used genetic algorithm to learn linkage intensity from limited data. The simulation results demonstrate that this algorithm is more suitable than the classic algorithm in the condition of sample shortage such as space shuttle’s fault diagnoisis.

Target threat estimation based on discrete dynamic Bayesian networks with small samples

The accuracy of target threat estimation has a great impact on command decision-making.The Bayesian network,as an effective way to deal with the problem of uncertainty,can be used to track the change of the target threat level.Unfortunately,the traditional discrete dynamic Bayesian network(DDBN)has the problems of poor parameter learning and poor reasoning accuracy in a small sample environment with partial prior information missing.Considering the finiteness and discreteness of DDBN parameters,a fuzzy k-nearest neighbor(KNN)algorithm based on correlation of feature quantities(CF-FKNN)is proposed for DDBN parameter learning.Firstly,the correlation between feature quantities is calculated,and then the KNN algorithm with fuzzy weight is introduced to fill the missing data.On this basis,a reasonable DDBN structure is constructed by using expert experience to complete DDBN parameter learning and reasoning.Simulation results show that the CF-FKNN algorithm can accurately fill in the data when the samples are seriously missing,and improve the effect of DDBN parameter learning in the case of serious sample missing.With the proposed method,the final target threat assessment results are reasonable,which meets the needs of engineering applications.

A Software Risk Analysis Model Using Bayesian Belief Network

The uncertainty during the period of software project development often brings huge risks to contractors and clients. If we can find an effective method to predict the cost and quality of software projects based on facts like the project character and two-side cooperating capability at the beginning of the project,we can reduce the risk. Bayesian Belief Network(BBN) is a good tool for analyzing uncertain consequences, but it is difficult to produce precise network structure and conditional probability table.In this paper,we built up network structure by Delphi method for conditional probability table learning,and learn update probability table and nodes’confidence levels continuously according to the application cases, which made the evaluation network have learning abilities, and evaluate the software development risk of organization more accurately.This paper also introduces EM algorithm, which will enhance the ability to produce hidden nodes caused by variant software projects.

Knowledge graph construction with structure and parameter learning for indoor scene design

We consider the problem of learning a representation of both spatial relations and dependencies between objects for indoor scene design.We propose a novel knowledge graph framework based on the entity-relation model for representation of facts in indoor scene design, and further develop a weaklysupervised algorithm for extracting the knowledge graph representation from a small dataset using both structure and parameter learning. The proposed framework is flexible, transferable, and readable. We present a variety of computer-aided indoor scene design applications using this representation, to show the usefulness and robustness of the proposed framework.

Application of machine learning algorithms for the evaluation of seismic soil liquefaction potential

This study investigates the performance of four machine learning(ML)algorithms to evaluate the earthquake-induced liquefaction potential of soil based on the cone penetration test field case history records using the Bayesian belief network(BBN)learning software Netica.The BBN structures that were developed by ML algorithms-K2,hill climbing(HC),tree augmented naive(TAN)Bayes,and Tabu search were adopted to perform parameter learning in Netica,thereby fixing the BBN models.The performance measure indexes,namely,overall accuracy(OA),precision,recall,F-measure,and area under the receiver operating characteristic curve,were used to evaluate the training and testing BBN models’performance and highlight the capability of the K2 and TAN Bayes models over the Tabu search and HC models.The sensitivity analysis results showed that the cone tip resistance and vertical effective stress are the most sensitive factors,whereas the mean grain size is the least sensitive factor in the prediction of seismic soil liquefaction potential.The results of this study can provide theoretical support for researchers in selecting appropriate ML algorithms and improving the predictive performance of seismic soil liquefaction potential models.

A new learning method using prior information of neural networks

In this paper, we present a new learning method using prior information for three-layered neural networks. Usually when neural networks are used for identification of systems, all of their weights are trained independently, without considering their interrelation of weight values. Thus the training results are not usually good. The reason for this is that each parameter has its influence on others during the learning. To overcome this problem, first, we give an exact mathematical equation that describes the relation between weight values given by a set of data conveying prior information. Then we present a new learning method that trains a part of the weights and calculates the others by using these exact mathematical equations. In almost all cases, this method keeps prior information given by a mathematical structure exactly during the learning. In addition, a learning method using prior information expressed by inequality is also presented. In any case, the degree of freedom of networks (the number of adjustable weights) is appropriately limited in order to speed up the learning and ensure small errors. Numerical computer simulation results are provided to support the present approaches.

Reasoning Disaster Chains with Bayesian Network Estimated Under Expert Prior Knowledge

With the acceleration of global climate change and urbanization,disaster chains are always connected to artificial systems like critical infrastructure.The complexity and uncertainty of the disaster chain development process and the severity of the consequences have brought great challenges to emergency decision makers.The Bayesian network(BN)was applied in this study to reason about disaster chain scenarios to support the choice of appropriate response strategies.To capture the interacting relationships among different factors,a scenario representation model of disaster chains was developed,followed by the determination of the BN structure.In deriving the conditional probability tables of the BN model,we found that,due to the lack of data and the significant uncertainty of disaster chains,parameter learning methodologies based on data or expert knowledge alone are insufficient.By integrating both sample data and expert knowledge with the maximum entropy principle,we proposed a parameter estimation algorithm under expert prior knowledge(PEUK).Taking the rainstorm disaster chain as an example,we demonstrated the superiority of the PEUK-built BN model over the traditional maximum a posterior(MAP)algorithm and the direct expert opinion elicitation method.The results also demonstrate the potential of our BN scenario reasoning paradigm to assist real-world disaster decisions.

Implicit Function Based Open-loop Analysis Method for Detecting the SSR Using Identified System Parameters

This paper proposes an implicit function based open-loop analysis method to detect the subsynchronous resonance(SSR),including asymmetric subsynchronous modal attraction(ASSMA)and asymmetric subsynchronous modal repulsion(ASSMR),of doubly-fed induction generator based wind farms(DFIG-WFs)penetrated power systems.As some important parameters of DFIG-WF are difficult to obtain,reinforcement learning and least squares method are applied to identify those important parameters.By predicting the location of closed-loop subsynchronous oscillation(SSO)modes based on the calculation of partial differentials of characteristic equation,both ASSMA and ASSMR can be found.The proposed method in this paper can select SSO modes which move to the right half complex planes as control parameters change.Besides,the proposed open-loop analysis method is adaptive to parameter uncertainty.Simulation studies are carried out on the 4-machine 11-bus power system to verify properties of the proposed method.