Most of the neural network architectures are based on human experience,which requires a long and tedious trial-and-error process.Neural architecture search(NAS)attempts to detect effective architectures without human ...Most of the neural network architectures are based on human experience,which requires a long and tedious trial-and-error process.Neural architecture search(NAS)attempts to detect effective architectures without human intervention.Evolutionary algorithms(EAs)for NAS can find better solutions than human-designed architectures by exploring a large search space for possible architectures.Using multiobjective EAs for NAS,optimal neural architectures that meet various performance criteria can be explored and discovered efficiently.Furthermore,hardware-accelerated NAS methods can improve the efficiency of the NAS.While existing reviews have mainly focused on different strategies to complete NAS,a few studies have explored the use of EAs for NAS.In this paper,we summarize and explore the use of EAs for NAS,as well as large-scale multiobjective optimization strategies and hardware-accelerated NAS methods.NAS performs well in healthcare applications,such as medical image analysis,classification of disease diagnosis,and health monitoring.EAs for NAS can automate the search process and optimize multiple objectives simultaneously in a given healthcare task.Deep neural network has been successfully used in healthcare,but it lacks interpretability.Medical data is highly sensitive,and privacy leaks are frequently reported in the healthcare industry.To solve these problems,in healthcare,we propose an interpretable neuroevolution framework based on federated learning to address search efficiency and privacy protection.Moreover,we also point out future research directions for evolutionary NAS.Overall,for researchers who want to use EAs to optimize NNs in healthcare,we analyze the advantages and disadvantages of doing so to provide detailed guidance,and propose an interpretable privacy-preserving framework for healthcare applications.展开更多
Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to ...Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.展开更多
Deep neural networks often outperform classical machine learning algorithms in solving real-world problems.However,designing better networks usually requires domain expertise and consumes significant time and com-puti...Deep neural networks often outperform classical machine learning algorithms in solving real-world problems.However,designing better networks usually requires domain expertise and consumes significant time and com-puting resources.Moreover,when the task changes,the original network architecture becomes outdated and requires redesigning.Thus,Neural Architecture Search(NAS)has gained attention as an effective approach to automatically generate optimal network architectures.Most NAS methods mainly focus on achieving high performance while ignoring architectural complexity.A myriad of research has revealed that network performance and structural complexity are often positively correlated.Nevertheless,complex network structures will bring enormous computing resources.To cope with this,we formulate the neural architecture search task as a multi-objective optimization problem,where an optimal architecture is learned by minimizing the classification error rate and the number of network parameters simultaneously.And then a decomposition-based multi-objective stochastic fractal search method is proposed to solve it.In view of the discrete property of the NAS problem,we discretize the stochastic fractal search step size so that the network architecture can be optimized more effectively.Additionally,two distinct update methods are employed in step size update stage to enhance the global and local search abilities adaptively.Furthermore,an information exchange mechanism between architectures is raised to accelerate the convergence process and improve the efficiency of the algorithm.Experimental studies show that the proposed algorithm has competitive performance comparable to many existing manual and automatic deep neural network generation approaches,which achieved a parameter-less and high-precision architecture with low-cost on each of the six benchmark datasets.展开更多
Recently,due to the availability of big data and the rapid growth of computing power,artificial intelligence(AI)has regained tremendous attention and investment.Machine learning(ML)approaches have been successfully ap...Recently,due to the availability of big data and the rapid growth of computing power,artificial intelligence(AI)has regained tremendous attention and investment.Machine learning(ML)approaches have been successfully applied to solve many problems in academia and in industry.Although the explosion of big data applications is driving the development of ML,it also imposes severe challenges of data processing speed and scalability on conventional computer systems.Computing platforms that are dedicatedly designed for AI applications have been considered,ranging from a complement to von Neumann platforms to a“must-have”and stand-alone technical solution.These platforms,which belong to a larger category named“domain-specific computing,”focus on specific customization for AI.In this article,we focus on summarizing the recent advances in accelerator designs for deep neural networks(DNNs)-that is,DNN accelerators.We discuss various architectures that support DNN executions in terms of computing units,dataflow optimization,targeted network topologies,architectures on emerging technologies,and accelerators for emerging applications.We also provide our visions on the future trend of AI chip designs.展开更多
How to recognize targets with similar appearances from remote sensing images(RSIs) effectively and efficiently has become a big challenge. Recently, convolutional neural network(CNN) is preferred in the target classif...How to recognize targets with similar appearances from remote sensing images(RSIs) effectively and efficiently has become a big challenge. Recently, convolutional neural network(CNN) is preferred in the target classification due to the powerful feature representation ability and better performance. However,the training and testing of CNN mainly rely on single machine.Single machine has its natural limitation and bottleneck in processing RSIs due to limited hardware resources and huge time consuming. Besides, overfitting is a challenge for the CNN model due to the unbalance between RSIs data and the model structure.When a model is complex or the training data is relatively small,overfitting occurs and leads to a poor predictive performance. To address these problems, a distributed CNN architecture for RSIs target classification is proposed, which dramatically increases the training speed of CNN and system scalability. It improves the storage ability and processing efficiency of RSIs. Furthermore,Bayesian regularization approach is utilized in order to initialize the weights of the CNN extractor, which increases the robustness and flexibility of the CNN model. It helps prevent the overfitting and avoid the local optima caused by limited RSI training images or the inappropriate CNN structure. In addition, considering the efficiency of the Na¨?ve Bayes classifier, a distributed Na¨?ve Bayes classifier is designed to reduce the training cost. Compared with other algorithms, the proposed system and method perform the best and increase the recognition accuracy. The results show that the distributed system framework and the proposed algorithms are suitable for RSIs target classification tasks.展开更多
In this paper,a novel method of ultra-lightweight convolution neural network(CNN)design based on neural architecture search(NAS)and knowledge distillation(KD)is proposed.It can realize the automatic construction of th...In this paper,a novel method of ultra-lightweight convolution neural network(CNN)design based on neural architecture search(NAS)and knowledge distillation(KD)is proposed.It can realize the automatic construction of the space target inverse synthetic aperture radar(ISAR)image recognition model with ultra-lightweight and high accuracy.This method introduces the NAS method into the radar image recognition for the first time,which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model(STIIARM).On this basis,the NAS model’s knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure(FSP)distillation method.Thus,the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided,and the ultralightweight STIIARM can be obtained.In the method,the Inverted Linear Bottleneck(ILB)and Inverted Residual Block(IRB)are firstly taken as each block’s basic structure in CNN.And the expansion ratio,output filter size,number of IRBs,and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space.Then,the recognition accuracy and computational complexity are taken as the objective function and constraint conditions,respectively,and the global optimization model of the CNN architecture search is established.Next,the simulated annealing(SA)algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly.After that,based on the three principles of similar block structure,the same corresponding channel number,and the minimum computational complexity,the more lightweight student model is designed,and the FSP matrix pairing between the NAS model and student model is completed.Finally,by minimizing the loss between the FSP matrix pairs of the NAS model and student model,the student model’s weight adjustment is completed.Thus the ultra-lightweight and high accuracy STIIARM is obtained.The proposed method’s effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.展开更多
This study presents a deep learning model for efficient intracranial hemorrhage(ICH)detection and subtype classification on non-contrast head computed tomography(CT)images.ICH refers to bleeding in the skull,leading t...This study presents a deep learning model for efficient intracranial hemorrhage(ICH)detection and subtype classification on non-contrast head computed tomography(CT)images.ICH refers to bleeding in the skull,leading to the most critical life-threatening health condition requiring rapid and accurate diagnosis.It is classified as intra-axial hemorrhage(intraventricular,intraparenchymal)and extra-axial hemorrhage(subdural,epidural,subarachnoid)based on the bleeding location inside the skull.Many computer-aided diagnoses(CAD)-based schemes have been proposed for ICH detection and classification at both slice and scan levels.However,these approaches performonly binary classification and suffer from a large number of parameters,which increase storage costs.Further,the accuracy of brain hemorrhage detection in existing models is significantly low for medically critical applications.To overcome these problems,a fast and efficient system for the automatic detection of ICH is needed.We designed a double-branch model based on xception architecture that extracts spatial and instant features,concatenates them,and creates the 3D spatial context(common feature vectors)fed to a decision tree classifier for final predictions.The data employed for the experimentation was gathered during the 2019 Radiologist Society of North America(RSNA)brain hemorrhage detection challenge.Our model outperformed benchmark models and achieved better accuracy in intraventricular(99.49%),subarachnoid(99.49%),intraparenchymal(99.10%),and subdural(98.09%)categories,thereby justifying the performance of the proposed double-branch xception architecture for ICH detection and classification.展开更多
Plant disease classification based on digital pictures is challenging.Machine learning approaches and plant image categorization technologies such as deep learning have been utilized to recognize,identify,and diagnose...Plant disease classification based on digital pictures is challenging.Machine learning approaches and plant image categorization technologies such as deep learning have been utilized to recognize,identify,and diagnose plant diseases in the previous decade.Increasing the yield quantity and quality of rice forming is an important cause for the paddy production countries.However,some diseases that are blocking the improvement in paddy production are considered as an ominous threat.Convolution Neural Network(CNN)has shown a remarkable performance in solving the early detection of paddy leaf diseases based on its images in the fast-growing era of science and technology.Nevertheless,the significant CNN architectures construction is dependent on expertise in a neural network and domain knowledge.This approach is time-consuming,and high computational resources are mandatory.In this research,we propose a novel method based on Mutant Particle swarm optimization(MUT-PSO)Algorithms to search for an optimum CNN architecture for Paddy leaf disease classification.Experimentation results show that Mutant Particle swarm optimization Convolution Neural Network(MUTPSO-CNN)can find optimumCNNarchitecture that offers better performance than existing hand-crafted CNN architectures in terms of accuracy,precision/recall,and execution time.展开更多
The dynamic working process of 52SFZ-140-207B type of hydraulic bumper isanalyzed. The modeling method using architecture-based neural networks is introduced. Using thismodeling method, the dynamic model of the hydrau...The dynamic working process of 52SFZ-140-207B type of hydraulic bumper isanalyzed. The modeling method using architecture-based neural networks is introduced. Using thismodeling method, the dynamic model of the hydraulic bumper is established; Based on this model thestructural parameters of the hydraulic bumper are optimized with Genetic algorithm. The result showsthat the performance of the dynamic model is close to that of the hydraulic bumper, and the dynamicperformance of the hydraulic bumper is improved through parameter optimization.展开更多
Side channel attacks(SCAs)on neural networks(NNs)are particularly efficient for retrieving secret information from NNs.We differentiate multiple types of threat scenarios regarding what kind of information is availabl...Side channel attacks(SCAs)on neural networks(NNs)are particularly efficient for retrieving secret information from NNs.We differentiate multiple types of threat scenarios regarding what kind of information is available before the attack and its purpose:recovering hyperparameters(the architecture)of the targeted NN,its weights(parameters),or its inputs.In this survey article,we consider the most relevant attacks to extract the architecture of CNNs.We also categorize SCAs,depending on access with respect to the victim:physical,local,or remote.Attacks targeting the architecture via local SCAs are most common.As of today,physical access seems necessary to retrieve the weights of an NN.We notably describe cache attacks,which are local SCAs aiming to extract the NN's underlying architecture.Few countermeasures have emerged;these are presented at the end of the survey.展开更多
Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the ...Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks(DNNs).However,NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS,and training DNNs is computationally intensive.To solve this major limitation of NAS,improving the computational efficiency is essential in the design of NAS.However,a systematic overview of computationally efficient NAS(CE-NAS)methods still lacks.To fill this gap,we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods,together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities.The remaining challenges and open research questions are also discussed,and promising research topics in this emerging field are suggested.展开更多
AutoML(Automated Machine Learning)is an emerging field that aims to automate the process of building machine learning models.AutoML emerged to increase productivity and efficiency by automating as much as possible the...AutoML(Automated Machine Learning)is an emerging field that aims to automate the process of building machine learning models.AutoML emerged to increase productivity and efficiency by automating as much as possible the inefficient work that occurs while repeating this process whenever machine learning is applied.In particular,research has been conducted for a long time on technologies that can effectively develop high-quality models by minimizing the intervention of model developers in the process from data preprocessing to algorithm selection and tuning.In this semantic review research,we summarize the data processing requirements for AutoML approaches and provide a detailed explanation.We place greater emphasis on neural architecture search(NAS)as it currently represents a highly popular sub-topic within the field of AutoML.NAS methods use machine learning algorithms to search through a large space of possible architectures and find the one that performs best on a given task.We provide a summary of the performance achieved by representative NAS algorithms on the CIFAR-10,CIFAR-100,ImageNet and wellknown benchmark datasets.Additionally,we delve into several noteworthy research directions in NAS methods including one/two-stage NAS,one-shot NAS and joint hyperparameter with architecture optimization.We discussed how the search space size and complexity in NAS can vary depending on the specific problem being addressed.To conclude,we examine several open problems(SOTA problems)within current AutoML methods that assure further investigation in future research.展开更多
Enterprise Information System management has become an increasingly vital factor for many firms. Several organizations have encountered problems when attempting to evaluate organizational performance. Measurement of p...Enterprise Information System management has become an increasingly vital factor for many firms. Several organizations have encountered problems when attempting to evaluate organizational performance. Measurement of performance metrics is a key challenge for a huge number of firms. In order to preserve relevance and adaptability in competitive markets, it has become essential to respond proactively to complex events through informed decision-making that is supported by technology. Therefore, the objective of this study was to apply neural networks to the modeling, simulation, and forecasting of the effects of the performance indicators of Enterprise Information Systems on the achievement of corporate objectives and value creation. A set of quantifiable and sizeable conditionally independent associations were derived using a simplified joint probability distribution technique. Bayesian Neural Networks were utilized to describe the link between random variables (features) and to concisely and easily specify the joint probability distribution. The research demonstrated that Bayesian networks could effectively explore complex logical linkages by employing probability to represent uncertainty and probabilistic rules;and by applying impact models from Bayesian taxonomies to achieve learning and reasoning processes.展开更多
基金supported in part by the National Natural Science Foundation of China (NSFC) under Grant No.61976242in part by the Natural Science Fund of Hebei Province for Distinguished Young Scholars under Grant No.F2021202010+2 种基金in part by the Fundamental Scientific Research Funds for Interdisciplinary Team of Hebei University of Technology under Grant No.JBKYTD2002funded by Science and Technology Project of Hebei Education Department under Grant No.JZX2023007supported by 2022 Interdisciplinary Postgraduate Training Program of Hebei University of Technology under Grant No.HEBUT-YXKJC-2022122.
文摘Most of the neural network architectures are based on human experience,which requires a long and tedious trial-and-error process.Neural architecture search(NAS)attempts to detect effective architectures without human intervention.Evolutionary algorithms(EAs)for NAS can find better solutions than human-designed architectures by exploring a large search space for possible architectures.Using multiobjective EAs for NAS,optimal neural architectures that meet various performance criteria can be explored and discovered efficiently.Furthermore,hardware-accelerated NAS methods can improve the efficiency of the NAS.While existing reviews have mainly focused on different strategies to complete NAS,a few studies have explored the use of EAs for NAS.In this paper,we summarize and explore the use of EAs for NAS,as well as large-scale multiobjective optimization strategies and hardware-accelerated NAS methods.NAS performs well in healthcare applications,such as medical image analysis,classification of disease diagnosis,and health monitoring.EAs for NAS can automate the search process and optimize multiple objectives simultaneously in a given healthcare task.Deep neural network has been successfully used in healthcare,but it lacks interpretability.Medical data is highly sensitive,and privacy leaks are frequently reported in the healthcare industry.To solve these problems,in healthcare,we propose an interpretable neuroevolution framework based on federated learning to address search efficiency and privacy protection.Moreover,we also point out future research directions for evolutionary NAS.Overall,for researchers who want to use EAs to optimize NNs in healthcare,we analyze the advantages and disadvantages of doing so to provide detailed guidance,and propose an interpretable privacy-preserving framework for healthcare applications.
基金supported in part by the National Key Research and Development Program of China under Grant 2019YFB2102102in part by the National Natural Science Foundations of China under Grant 62176094 and Grant 61873097+2 种基金in part by the Key‐Area Research and Development of Guangdong Province under Grant 2020B010166002in part by the Guangdong Natural Science Foundation Research Team under Grant 2018B030312003in part by the Guangdong‐Hong Kong Joint Innovation Platform under Grant 2018B050502006.
文摘Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.
基金supported by the China Postdoctoral Science Foundation Funded Project(Grant Nos.2017M613054 and 2017M613053)the Shaanxi Postdoctoral Science Foundation Funded Project(Grant No.2017BSHYDZZ33)the National Science Foundation of China(Grant No.62102239).
文摘Deep neural networks often outperform classical machine learning algorithms in solving real-world problems.However,designing better networks usually requires domain expertise and consumes significant time and com-puting resources.Moreover,when the task changes,the original network architecture becomes outdated and requires redesigning.Thus,Neural Architecture Search(NAS)has gained attention as an effective approach to automatically generate optimal network architectures.Most NAS methods mainly focus on achieving high performance while ignoring architectural complexity.A myriad of research has revealed that network performance and structural complexity are often positively correlated.Nevertheless,complex network structures will bring enormous computing resources.To cope with this,we formulate the neural architecture search task as a multi-objective optimization problem,where an optimal architecture is learned by minimizing the classification error rate and the number of network parameters simultaneously.And then a decomposition-based multi-objective stochastic fractal search method is proposed to solve it.In view of the discrete property of the NAS problem,we discretize the stochastic fractal search step size so that the network architecture can be optimized more effectively.Additionally,two distinct update methods are employed in step size update stage to enhance the global and local search abilities adaptively.Furthermore,an information exchange mechanism between architectures is raised to accelerate the convergence process and improve the efficiency of the algorithm.Experimental studies show that the proposed algorithm has competitive performance comparable to many existing manual and automatic deep neural network generation approaches,which achieved a parameter-less and high-precision architecture with low-cost on each of the six benchmark datasets.
基金the National Science Foundations(NSFs)(1822085,1725456,1816833,1500848,1719160,and 1725447)the NSF Computing and Communication Foundations(1740352)+1 种基金the Nanoelectronics COmputing REsearch Program in the Semiconductor Research Corporation(NC-2766-A)the Center for Research in Intelligent Storage and Processing-in-Memory,one of six centers in the Joint University Microelectronics Program,a SRC program sponsored by Defense Advanced Research Projects Agency.
文摘Recently,due to the availability of big data and the rapid growth of computing power,artificial intelligence(AI)has regained tremendous attention and investment.Machine learning(ML)approaches have been successfully applied to solve many problems in academia and in industry.Although the explosion of big data applications is driving the development of ML,it also imposes severe challenges of data processing speed and scalability on conventional computer systems.Computing platforms that are dedicatedly designed for AI applications have been considered,ranging from a complement to von Neumann platforms to a“must-have”and stand-alone technical solution.These platforms,which belong to a larger category named“domain-specific computing,”focus on specific customization for AI.In this article,we focus on summarizing the recent advances in accelerator designs for deep neural networks(DNNs)-that is,DNN accelerators.We discuss various architectures that support DNN executions in terms of computing units,dataflow optimization,targeted network topologies,architectures on emerging technologies,and accelerators for emerging applications.We also provide our visions on the future trend of AI chip designs.
基金supported by the National Natural Science Foundation of China(U1435220)
文摘How to recognize targets with similar appearances from remote sensing images(RSIs) effectively and efficiently has become a big challenge. Recently, convolutional neural network(CNN) is preferred in the target classification due to the powerful feature representation ability and better performance. However,the training and testing of CNN mainly rely on single machine.Single machine has its natural limitation and bottleneck in processing RSIs due to limited hardware resources and huge time consuming. Besides, overfitting is a challenge for the CNN model due to the unbalance between RSIs data and the model structure.When a model is complex or the training data is relatively small,overfitting occurs and leads to a poor predictive performance. To address these problems, a distributed CNN architecture for RSIs target classification is proposed, which dramatically increases the training speed of CNN and system scalability. It improves the storage ability and processing efficiency of RSIs. Furthermore,Bayesian regularization approach is utilized in order to initialize the weights of the CNN extractor, which increases the robustness and flexibility of the CNN model. It helps prevent the overfitting and avoid the local optima caused by limited RSI training images or the inappropriate CNN structure. In addition, considering the efficiency of the Na¨?ve Bayes classifier, a distributed Na¨?ve Bayes classifier is designed to reduce the training cost. Compared with other algorithms, the proposed system and method perform the best and increase the recognition accuracy. The results show that the distributed system framework and the proposed algorithms are suitable for RSIs target classification tasks.
文摘In this paper,a novel method of ultra-lightweight convolution neural network(CNN)design based on neural architecture search(NAS)and knowledge distillation(KD)is proposed.It can realize the automatic construction of the space target inverse synthetic aperture radar(ISAR)image recognition model with ultra-lightweight and high accuracy.This method introduces the NAS method into the radar image recognition for the first time,which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model(STIIARM).On this basis,the NAS model’s knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure(FSP)distillation method.Thus,the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided,and the ultralightweight STIIARM can be obtained.In the method,the Inverted Linear Bottleneck(ILB)and Inverted Residual Block(IRB)are firstly taken as each block’s basic structure in CNN.And the expansion ratio,output filter size,number of IRBs,and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space.Then,the recognition accuracy and computational complexity are taken as the objective function and constraint conditions,respectively,and the global optimization model of the CNN architecture search is established.Next,the simulated annealing(SA)algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly.After that,based on the three principles of similar block structure,the same corresponding channel number,and the minimum computational complexity,the more lightweight student model is designed,and the FSP matrix pairing between the NAS model and student model is completed.Finally,by minimizing the loss between the FSP matrix pairs of the NAS model and student model,the student model’s weight adjustment is completed.Thus the ultra-lightweight and high accuracy STIIARM is obtained.The proposed method’s effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.
文摘This study presents a deep learning model for efficient intracranial hemorrhage(ICH)detection and subtype classification on non-contrast head computed tomography(CT)images.ICH refers to bleeding in the skull,leading to the most critical life-threatening health condition requiring rapid and accurate diagnosis.It is classified as intra-axial hemorrhage(intraventricular,intraparenchymal)and extra-axial hemorrhage(subdural,epidural,subarachnoid)based on the bleeding location inside the skull.Many computer-aided diagnoses(CAD)-based schemes have been proposed for ICH detection and classification at both slice and scan levels.However,these approaches performonly binary classification and suffer from a large number of parameters,which increase storage costs.Further,the accuracy of brain hemorrhage detection in existing models is significantly low for medically critical applications.To overcome these problems,a fast and efficient system for the automatic detection of ICH is needed.We designed a double-branch model based on xception architecture that extracts spatial and instant features,concatenates them,and creates the 3D spatial context(common feature vectors)fed to a decision tree classifier for final predictions.The data employed for the experimentation was gathered during the 2019 Radiologist Society of North America(RSNA)brain hemorrhage detection challenge.Our model outperformed benchmark models and achieved better accuracy in intraventricular(99.49%),subarachnoid(99.49%),intraparenchymal(99.10%),and subdural(98.09%)categories,thereby justifying the performance of the proposed double-branch xception architecture for ICH detection and classification.
基金The authors received funding source for this research activity under Multi-Disciplinary Research(MDR)Grant Vot H483 from Research Management Centre(RMC)office,Universiti Tun Hussein Onn Malaysia(UTHM).
文摘Plant disease classification based on digital pictures is challenging.Machine learning approaches and plant image categorization technologies such as deep learning have been utilized to recognize,identify,and diagnose plant diseases in the previous decade.Increasing the yield quantity and quality of rice forming is an important cause for the paddy production countries.However,some diseases that are blocking the improvement in paddy production are considered as an ominous threat.Convolution Neural Network(CNN)has shown a remarkable performance in solving the early detection of paddy leaf diseases based on its images in the fast-growing era of science and technology.Nevertheless,the significant CNN architectures construction is dependent on expertise in a neural network and domain knowledge.This approach is time-consuming,and high computational resources are mandatory.In this research,we propose a novel method based on Mutant Particle swarm optimization(MUT-PSO)Algorithms to search for an optimum CNN architecture for Paddy leaf disease classification.Experimentation results show that Mutant Particle swarm optimization Convolution Neural Network(MUTPSO-CNN)can find optimumCNNarchitecture that offers better performance than existing hand-crafted CNN architectures in terms of accuracy,precision/recall,and execution time.
文摘The dynamic working process of 52SFZ-140-207B type of hydraulic bumper isanalyzed. The modeling method using architecture-based neural networks is introduced. Using thismodeling method, the dynamic model of the hydraulic bumper is established; Based on this model thestructural parameters of the hydraulic bumper are optimized with Genetic algorithm. The result showsthat the performance of the dynamic model is close to that of the hydraulic bumper, and the dynamicperformance of the hydraulic bumper is improved through parameter optimization.
文摘Side channel attacks(SCAs)on neural networks(NNs)are particularly efficient for retrieving secret information from NNs.We differentiate multiple types of threat scenarios regarding what kind of information is available before the attack and its purpose:recovering hyperparameters(the architecture)of the targeted NN,its weights(parameters),or its inputs.In this survey article,we consider the most relevant attacks to extract the architecture of CNNs.We also categorize SCAs,depending on access with respect to the victim:physical,local,or remote.Attacks targeting the architecture via local SCAs are most common.As of today,physical access seems necessary to retrieve the weights of an NN.We notably describe cache attacks,which are local SCAs aiming to extract the NN's underlying architecture.Few countermeasures have emerged;these are presented at the end of the survey.
基金This work was supported by a Ulucu PhD studentshipY.Jin is funded by an Alexander von Humboldt Professorship for Artificial Intelligence endowed by the German Federal Ministry of Education and Research.
文摘Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks(DNNs).However,NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS,and training DNNs is computationally intensive.To solve this major limitation of NAS,improving the computational efficiency is essential in the design of NAS.However,a systematic overview of computationally efficient NAS(CE-NAS)methods still lacks.To fill this gap,we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods,together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities.The remaining challenges and open research questions are also discussed,and promising research topics in this emerging field are suggested.
文摘AutoML(Automated Machine Learning)is an emerging field that aims to automate the process of building machine learning models.AutoML emerged to increase productivity and efficiency by automating as much as possible the inefficient work that occurs while repeating this process whenever machine learning is applied.In particular,research has been conducted for a long time on technologies that can effectively develop high-quality models by minimizing the intervention of model developers in the process from data preprocessing to algorithm selection and tuning.In this semantic review research,we summarize the data processing requirements for AutoML approaches and provide a detailed explanation.We place greater emphasis on neural architecture search(NAS)as it currently represents a highly popular sub-topic within the field of AutoML.NAS methods use machine learning algorithms to search through a large space of possible architectures and find the one that performs best on a given task.We provide a summary of the performance achieved by representative NAS algorithms on the CIFAR-10,CIFAR-100,ImageNet and wellknown benchmark datasets.Additionally,we delve into several noteworthy research directions in NAS methods including one/two-stage NAS,one-shot NAS and joint hyperparameter with architecture optimization.We discussed how the search space size and complexity in NAS can vary depending on the specific problem being addressed.To conclude,we examine several open problems(SOTA problems)within current AutoML methods that assure further investigation in future research.
文摘Enterprise Information System management has become an increasingly vital factor for many firms. Several organizations have encountered problems when attempting to evaluate organizational performance. Measurement of performance metrics is a key challenge for a huge number of firms. In order to preserve relevance and adaptability in competitive markets, it has become essential to respond proactively to complex events through informed decision-making that is supported by technology. Therefore, the objective of this study was to apply neural networks to the modeling, simulation, and forecasting of the effects of the performance indicators of Enterprise Information Systems on the achievement of corporate objectives and value creation. A set of quantifiable and sizeable conditionally independent associations were derived using a simplified joint probability distribution technique. Bayesian Neural Networks were utilized to describe the link between random variables (features) and to concisely and easily specify the joint probability distribution. The research demonstrated that Bayesian networks could effectively explore complex logical linkages by employing probability to represent uncertainty and probabilistic rules;and by applying impact models from Bayesian taxonomies to achieve learning and reasoning processes.