Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood v...Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood vessels and the herculean task involved in glaucoma detection,the exactly affected site of the optic disc of whether small or big size cup,is deemed challenging.Spatially Based Ellipse Fitting Curve Model(SBEFCM)classification is suggested based on the Ensemble for a reliable diagnosis of Glaucomain theOptic Cup(OC)and Optic Disc(OD)boundary correspondingly.This research deploys the Ensemble Convolutional Neural Network(CNN)classification for classifying Glaucoma or Diabetes Retinopathy(DR).The detection of the boundary between the OC and the OD is performed by the SBEFCM,which is the latest weighted ellipse fitting model.The SBEFCM that enhances and widens the multi-ellipse fitting technique is proposed here.There is a preprocessing of input fundus image besides segmentation of blood vessels to avoid interlacing surrounding tissues and blood vessels.The ascertaining of OCandODboundary,which characterizedmany output factors for glaucoma detection,has been developed by EnsembleCNNclassification,which includes detecting sensitivity,specificity,precision,andArea Under the receiver operating characteristic Curve(AUC)values accurately by an innovative SBEFCM.In terms of contrast,the proposed Ensemble CNNsignificantly outperformed the current methods.展开更多
This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed fo...This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed for the supervised learning model. Based on the bias-variance trade-off, both the single neural network approaches and the neural network ensemble approaches are overviewed, and problems with the existing approaches are indicated. Finally, the paper concludes with specifying potential future research directions.展开更多
There has been many methods in constructing neural network (NN) ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constru...There has been many methods in constructing neural network (NN) ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constructing NN ensembles, it follows the two steps, first training component networks, and then combining them. As the two steps being independent, an assumption is used to facilitate interactions among NNs during the training stage. This paper presents a compact ensemble method which integrates the two steps of ensemble construction into one step by attempting to train individual NNs in an ensemble and weigh the individual members adaptively according to their individual performance in the same learning process. This provides an opportunity for the individual NNs to interact with each other based on their real contributions to the ensemble. The classification performance of NN compact ensemble (NNCE) was validated through some benchmark problems in machine learning, including Australian credit card assessment, pima Indians diabetes, heart disease, breast cancer and glass. Compared with other ensembles, the classification error rate of NNCE can be decreased by 0.45% to 68%. In addition, the NNCE was applied to fault diagnosis for rolling element bearing. The 11 time-domain statistical features are extracted as the properties of data, and the NNCE is employed to classify the data. With the results of several experiments, the compact ensemble method is shown to give good generalization performance. The compact ensemble method can recognize the different fault types and various fault degrees of the same fault type.展开更多
It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental informat...It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.展开更多
Neural network ensemble based on rough sets reduct is proposed to decrease the computational complexity of conventional ensemble feature selection algorithm. First, a dynamic reduction technology combining genetic alg...Neural network ensemble based on rough sets reduct is proposed to decrease the computational complexity of conventional ensemble feature selection algorithm. First, a dynamic reduction technology combining genetic algorithm with resampling method is adopted to obtain reducts with good generalization ability. Second, Multiple BP neural networks based on different reducts are built as base classifiers. According to the idea of selective ensemble, the neural network ensemble with best generalization ability can be found by search strategies. Finally, classification based on neural network ensemble is implemented by combining the predictions of component networks with voting. The method has been verified in the experiment of remote sensing image and five UCI datasets classification. Compared with conventional ensemble feature selection algorithms, it costs less time and lower computing complexity, and the classification accuracy is satisfactory.展开更多
Many industrial products are normally processed through multiple manufacturing process stages before it becomes a final product.Statistical process control techniques often utilize standard Shewhart control charts to ...Many industrial products are normally processed through multiple manufacturing process stages before it becomes a final product.Statistical process control techniques often utilize standard Shewhart control charts to monitor these process stages.If the process stages are independent,this is a meaningful procedure.However,they are not independent in many manufacturing scenarios.The standard Shewhart control charts can not provide the information to determine which process stage or group of process stages has caused the problems(i.e.,standard Shewhart control charts could not diagnose dependent manufacturing process stages).This study proposes a selective neural network ensemble-based cause-selecting system of control charts to monitor these process stages and distinguish incoming quality problems and problems in the current stage of a manufacturing process.Numerical results show that the proposed method is an improvement over the use of separate Shewhart control chart for each of dependent process stages,and even ordinary quality practitioners who lack of expertise in theoretical analysis can implement regression estimation and neural computing readily.展开更多
This paper describes negative correlation learning for designing neural network ensembles. Negative correlation learning has been firstly analysed in terms of minimising mutual information on a regression task. By min...This paper describes negative correlation learning for designing neural network ensembles. Negative correlation learning has been firstly analysed in terms of minimising mutual information on a regression task. By minimising the mutual information between variables extracted by two neural networks, they are forced to convey different information a-bout some features of their input. Based on the decision boundaries and correct response sets, negative correlation learning has been further studied on two pattern classification problems. The purpose of examining the decision boundaries and the correct response sets is not only to illustrate the learning behavior of negative correlation learning, but also to cast light on how to design more effective neural network ensembles. The experimental results showed the decision boundary of the trained neural network ensemble by negative correlation learning is almost as good as the optimum decision boundary.展开更多
With the advancement of satellite technology,a considerable amount of very high-resolution imagery has become available to be used for the Land Cover and Land Use(LCLU)classification task aiming to categorize remotely...With the advancement of satellite technology,a considerable amount of very high-resolution imagery has become available to be used for the Land Cover and Land Use(LCLU)classification task aiming to categorize remotely sensed images based on their semantic content.Recently,Deep Neural Networks(DNNs)have been widely used for different applications in the field of remote sensing and they have profound impacts;however,improvement of the generalizability and robustness of the DNNs needs to be progressed further to achieve higher accuracy for a variety of sensing geometries and categories.We address this problem by deploying three different Deep Neural Network Ensemble(DNNE)methods and creating a comparative analysis for the LCLU classification task.DNNE enables improvement of the performance of DNNs by ensuring the diversity of the models that are combined.Thus,enhances the generalizability of the models and produces more robust and generalizable outcomes for LCLU classification tasks.The experimental results on NWPU-RESISC45 and AID datasets demonstrate that utilizing the aggregated information from multiple DNNs leads to an increase in classification performance,achieves state-of-the-art,and promotes researchers to make use of DNNE.展开更多
After the consideration of the nonlinear nature changes of monsoon index,and the subjective determination of network structure in traditional artificial neural network prediction modeling,monthly and seasonal monsoon ...After the consideration of the nonlinear nature changes of monsoon index,and the subjective determination of network structure in traditional artificial neural network prediction modeling,monthly and seasonal monsoon intensity index prediction is studied in this paper by using nonlinear genetic neural network ensemble prediction(GNNEP)modeling.It differs from traditional prediction modeling in the following aspects: (1)Input factors of the GNNEP model of monsoon index were selected from a large quantity of preceding period high correlation factors,such as monthly sea temperature fields,monthly 500-hPa air temperature fields,monthly 200-hPa geopotential height fields,etc.,and they were also highly information-condensed and system dimensionality-reduced by using the empirical orthogonal function(EOF)method,which effectively condensed the useful information of predictors and therefore controlled the size of network structure of the GNNEP model.(2)In the input design of the GNNEP model,a mean generating function(MGF)series of predictand(monsoon index)was added as an input factor;the contrast analysis of results of predic- tion experiments by a physical variable predictor-predictand MGF GNNEP model and a physical variable predictor GNNEP model shows that the incorporation of the periodical variation of predictand(monsoon index)is very effective in improving the prediction of monsoon index.(3)Different from the traditional neural network modeling,the GNNEP modeling is able to objectively determine the network structure of the GNNNEP model,and the model constructed has a better generalization capability.In the case of identical predictors,prediction modeling samples,and independent prediction samples,the prediction accuracy of our GNNEP model combined with the system dimensionality reduction technique of predictors is clearly higher than that of the traditional stepwise regression model using the traditional treatment technique of predictors,suggesting that the GNNEP model opens up a vast range of possibilities for operational weather prediction.展开更多
Evolutionary computation has experienced a tremendous growth in the last decade in both theoretical analyses and industrial applications. Its scope has evolved beyond its original meaning of "biological evolution" t...Evolutionary computation has experienced a tremendous growth in the last decade in both theoretical analyses and industrial applications. Its scope has evolved beyond its original meaning of "biological evolution" toward a wide variety of nature inspired computational algorithms and techniques, including evolutionary, neural, ecological, social and economical computation, etc, in a unified framework. Many research topics in evolutionary computation nowadays are not necessarily "evolutionary". This paper provides an overview of some recent advances in evolutionary computation that have been made in CERCIA at the University of Birmingham, UK. It covers a wide range of topics in optimization, learning and design using evolutionary approaches and techniques, and theoretical results in the computational time complexity of evolutionary algorithms. Some issues related to future development of evolutionary computation are also discussed.展开更多
文摘Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood vessels and the herculean task involved in glaucoma detection,the exactly affected site of the optic disc of whether small or big size cup,is deemed challenging.Spatially Based Ellipse Fitting Curve Model(SBEFCM)classification is suggested based on the Ensemble for a reliable diagnosis of Glaucomain theOptic Cup(OC)and Optic Disc(OD)boundary correspondingly.This research deploys the Ensemble Convolutional Neural Network(CNN)classification for classifying Glaucoma or Diabetes Retinopathy(DR).The detection of the boundary between the OC and the OD is performed by the SBEFCM,which is the latest weighted ellipse fitting model.The SBEFCM that enhances and widens the multi-ellipse fitting technique is proposed here.There is a preprocessing of input fundus image besides segmentation of blood vessels to avoid interlacing surrounding tissues and blood vessels.The ascertaining of OCandODboundary,which characterizedmany output factors for glaucoma detection,has been developed by EnsembleCNNclassification,which includes detecting sensitivity,specificity,precision,andArea Under the receiver operating characteristic Curve(AUC)values accurately by an innovative SBEFCM.In terms of contrast,the proposed Ensemble CNNsignificantly outperformed the current methods.
基金Supported by the National Natural Science Foundation of China(60133010)
文摘This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed for the supervised learning model. Based on the bias-variance trade-off, both the single neural network approaches and the neural network ensemble approaches are overviewed, and problems with the existing approaches are indicated. Finally, the paper concludes with specifying potential future research directions.
基金supported by National Natural Science Foundation of China(Grant No.50575179)National Hi-tech Research and Development Program of China(863 Program,Grant No.2006AA04Z420)
文摘There has been many methods in constructing neural network (NN) ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constructing NN ensembles, it follows the two steps, first training component networks, and then combining them. As the two steps being independent, an assumption is used to facilitate interactions among NNs during the training stage. This paper presents a compact ensemble method which integrates the two steps of ensemble construction into one step by attempting to train individual NNs in an ensemble and weigh the individual members adaptively according to their individual performance in the same learning process. This provides an opportunity for the individual NNs to interact with each other based on their real contributions to the ensemble. The classification performance of NN compact ensemble (NNCE) was validated through some benchmark problems in machine learning, including Australian credit card assessment, pima Indians diabetes, heart disease, breast cancer and glass. Compared with other ensembles, the classification error rate of NNCE can be decreased by 0.45% to 68%. In addition, the NNCE was applied to fault diagnosis for rolling element bearing. The 11 time-domain statistical features are extracted as the properties of data, and the NNCE is employed to classify the data. With the results of several experiments, the compact ensemble method is shown to give good generalization performance. The compact ensemble method can recognize the different fault types and various fault degrees of the same fault type.
基金supported by the National Key Research and Development Program of China under Grant 2019YFC1511401the National Natural Science Foundation of China under Grant 62173038 and 61103157+1 种基金Science Foundation for Young Scholars of Tobacco Research Institute of Chinese Academy of Agricultural Sciences under Grant 2021B05Key Scientific and Tech-nological Research and Development Project of China National Tobacco Corporation under Grant 110202102007.
文摘It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.
基金supported by the National High-Tech Research and Development Plan of China (No.2007AA04Z224)the National Natural Science Foundation of China (No.60775047, 60835004)
文摘Neural network ensemble based on rough sets reduct is proposed to decrease the computational complexity of conventional ensemble feature selection algorithm. First, a dynamic reduction technology combining genetic algorithm with resampling method is adopted to obtain reducts with good generalization ability. Second, Multiple BP neural networks based on different reducts are built as base classifiers. According to the idea of selective ensemble, the neural network ensemble with best generalization ability can be found by search strategies. Finally, classification based on neural network ensemble is implemented by combining the predictions of component networks with voting. The method has been verified in the experiment of remote sensing image and five UCI datasets classification. Compared with conventional ensemble feature selection algorithms, it costs less time and lower computing complexity, and the classification accuracy is satisfactory.
基金supported in part by the National Natural Science Foundation of China(No.51775279)the Fundamental Research Funds for the Central Universities(Nos. 1005-YAH15055,NS2017034)+2 种基金the China Postdoctoral Science Foundation(No.2016M591838)the Natural Science Foundation of Jiangsu Province (No.BK20150745)the Postdoctoral Science Foundation of of Jiangsu Province(No.1501024C)
文摘Many industrial products are normally processed through multiple manufacturing process stages before it becomes a final product.Statistical process control techniques often utilize standard Shewhart control charts to monitor these process stages.If the process stages are independent,this is a meaningful procedure.However,they are not independent in many manufacturing scenarios.The standard Shewhart control charts can not provide the information to determine which process stage or group of process stages has caused the problems(i.e.,standard Shewhart control charts could not diagnose dependent manufacturing process stages).This study proposes a selective neural network ensemble-based cause-selecting system of control charts to monitor these process stages and distinguish incoming quality problems and problems in the current stage of a manufacturing process.Numerical results show that the proposed method is an improvement over the use of separate Shewhart control chart for each of dependent process stages,and even ordinary quality practitioners who lack of expertise in theoretical analysis can implement regression estimation and neural computing readily.
基金Supported by the National Natural Science Foundation of China(60133010)
文摘This paper describes negative correlation learning for designing neural network ensembles. Negative correlation learning has been firstly analysed in terms of minimising mutual information on a regression task. By minimising the mutual information between variables extracted by two neural networks, they are forced to convey different information a-bout some features of their input. Based on the decision boundaries and correct response sets, negative correlation learning has been further studied on two pattern classification problems. The purpose of examining the decision boundaries and the correct response sets is not only to illustrate the learning behavior of negative correlation learning, but also to cast light on how to design more effective neural network ensembles. The experimental results showed the decision boundary of the trained neural network ensemble by negative correlation learning is almost as good as the optimum decision boundary.
基金supported by The Scientific and Technological Research Council of Turkey(TÜBİTAK)under the 2210/C Scholarship Program in the Priority Fields in Science and Technology。
文摘With the advancement of satellite technology,a considerable amount of very high-resolution imagery has become available to be used for the Land Cover and Land Use(LCLU)classification task aiming to categorize remotely sensed images based on their semantic content.Recently,Deep Neural Networks(DNNs)have been widely used for different applications in the field of remote sensing and they have profound impacts;however,improvement of the generalizability and robustness of the DNNs needs to be progressed further to achieve higher accuracy for a variety of sensing geometries and categories.We address this problem by deploying three different Deep Neural Network Ensemble(DNNE)methods and creating a comparative analysis for the LCLU classification task.DNNE enables improvement of the performance of DNNs by ensuring the diversity of the models that are combined.Thus,enhances the generalizability of the models and produces more robust and generalizable outcomes for LCLU classification tasks.The experimental results on NWPU-RESISC45 and AID datasets demonstrate that utilizing the aggregated information from multiple DNNs leads to an increase in classification performance,achieves state-of-the-art,and promotes researchers to make use of DNNE.
基金the New Technology Extension Project of China Meteorological Administration under Grant No.GMATG2008M49the National Natural Science Foundation of China under Grant No.40675023
文摘After the consideration of the nonlinear nature changes of monsoon index,and the subjective determination of network structure in traditional artificial neural network prediction modeling,monthly and seasonal monsoon intensity index prediction is studied in this paper by using nonlinear genetic neural network ensemble prediction(GNNEP)modeling.It differs from traditional prediction modeling in the following aspects: (1)Input factors of the GNNEP model of monsoon index were selected from a large quantity of preceding period high correlation factors,such as monthly sea temperature fields,monthly 500-hPa air temperature fields,monthly 200-hPa geopotential height fields,etc.,and they were also highly information-condensed and system dimensionality-reduced by using the empirical orthogonal function(EOF)method,which effectively condensed the useful information of predictors and therefore controlled the size of network structure of the GNNEP model.(2)In the input design of the GNNEP model,a mean generating function(MGF)series of predictand(monsoon index)was added as an input factor;the contrast analysis of results of predic- tion experiments by a physical variable predictor-predictand MGF GNNEP model and a physical variable predictor GNNEP model shows that the incorporation of the periodical variation of predictand(monsoon index)is very effective in improving the prediction of monsoon index.(3)Different from the traditional neural network modeling,the GNNEP modeling is able to objectively determine the network structure of the GNNNEP model,and the model constructed has a better generalization capability.In the case of identical predictors,prediction modeling samples,and independent prediction samples,the prediction accuracy of our GNNEP model combined with the system dimensionality reduction technique of predictors is clearly higher than that of the traditional stepwise regression model using the traditional treatment technique of predictors,suggesting that the GNNEP model opens up a vast range of possibilities for operational weather prediction.
基金This work is partially supported by the National Natural Science Foundation of China (Grant No. 60428202), and the Advantage West Midlands, UK.
文摘Evolutionary computation has experienced a tremendous growth in the last decade in both theoretical analyses and industrial applications. Its scope has evolved beyond its original meaning of "biological evolution" toward a wide variety of nature inspired computational algorithms and techniques, including evolutionary, neural, ecological, social and economical computation, etc, in a unified framework. Many research topics in evolutionary computation nowadays are not necessarily "evolutionary". This paper provides an overview of some recent advances in evolutionary computation that have been made in CERCIA at the University of Birmingham, UK. It covers a wide range of topics in optimization, learning and design using evolutionary approaches and techniques, and theoretical results in the computational time complexity of evolutionary algorithms. Some issues related to future development of evolutionary computation are also discussed.