Improved gravitational search algorithm based on free search differential evolution

This paper presents an improved gravitational search algorithm （IGSA） as a hybridization of a relatively recent evolutionary algorithm called gravitational search algorithm （GSA）, with the free search differential evolution （FSDE）. This combination incorporates FSDE into the optimization process of GSA with an attempt to avoid the premature convergence in GSA. This strategy makes full use of the exploration ability of GSA and the exploitation ability of FSDE. IGSA is tested on a suite of benchmark functions. The experimental results demonstrate the good performance of IGSA.

Enhanced Differentiable Architecture Search Based on Asymptotic Regularization

In differentiable search architecture search methods,a more efficient search space design can significantly improve the performance of the searched architecture,thus requiring people to carefully define the search space with different complexity according to various operations.Meanwhile rationalizing the search strategies to explore the well-defined search space will further improve the speed and efficiency of architecture search.With this in mind,we propose a faster and more efficient differentiable architecture search method,AllegroNAS.Firstly,we introduce a more efficient search space enriched by the introduction of two redefined convolution modules.Secondly,we utilize a more efficient architectural parameter regularization method,mitigating the overfitting problem during the search process and reducing the error brought about by gradient approximation.Meanwhile,we introduce a natural exponential cosine annealing method to make the learning rate of the neural network training process more suitable for the search procedure.Moreover,group convolution and data augmentation are employed to reduce the computational cost.Finally,through extensive experiments on several public datasets,we demonstrate that our method can more swiftly search for better-performing neural network architectures in a more efficient search space,thus validating the effectiveness of our approach.

NAS-HR:Neural architecture search for heart rate estimation from face videos

Background In anticipation of its great potential application to natural human-computer interaction and health monitoring,heart-rate(HR)estimation based on remote photoplethysmography has recently attracted increasing research attention.Whereas the recent deep-learning-based HR estimation methods have achieved promising performance,their computational costs remain high,particularly in mobile-computing scenarios.Methods We propose a neural architecture search approach for HR estimation to automatically search a lightweight network that can achieve even higher accuracy than a complex network while reducing the computational cost.First,we define the regions of interests based on face landmarks and then extract the raw temporal pulse signals from the R,G,and B channels in each ROI.Then,pulse-related signals are extracted using a plane-orthogonal-to-skin algorithm,which are combined with the R and G channel signals to create a spatial-temporal map.Finally,a differentiable architecture search approach is used for the network-structure search.Results Compared with the state-of-the-art methods on the public-domain VIPL-HR and PURE databases,our method achieves better HR estimation performance in terms of several evaluation metrics while requiring a much lower computational cost1.

Adaptive probabilistic load forecasting for individual buildings

Building-level load forecasting has become essential with the support of fine-grained data collected by widely deployed smart meters.It acts as a basis for arranging distributed energy resources,implementing demand response,etc.Compared to aggre-gated-level load,the electric load of an individual building is more stochastic and thus spawns many probabilistic forecasting meth-ods.Many of them resort to artificial neural networks(ANN)to build forecasting models.However,a well-designed forecasting model for one building may not be suitable for others,and manually designing and tuning optimal forecasting models for various buildings are tedious and time-consuming.This paper proposes an adaptive probabilistic load forecasting model to automatically generate high-performance NN structures for different buildings and produce quantile forecasts for future loads.Specifically,we cascade the long short term memory(LSTM)layer with the adjusted Differential ArchiTecture Search(DARTS)cell and use the pinball loss function to guide the model during the improved model fitting process.A case study on an open dataset shows that our proposed model has superior performance and adaptivity over the state-of-the-art static neural network model.Besides,the improved fitting process of DARTS is proved to be more time-efficient than the original one.

DynamicRetriever:A Pre-trained Model-based IR System Without an Explicit Index

Web search provides a promising way for people to obtain information and has been extensively studied.With the surge of deep learning and large-scale pre-training techniques,various neural information retrieval models are proposed,and they have demonstrated the power for improving search(especially,the ranking)quality.All these existing search methods follow a common paradigm,i.e.,index-retrieve-rerank,where they first build an index of all documents based on document terms(i.e.,sparse inverted index)or representation vectors(i.e.,dense vector index),then retrieve and rerank retrieved documents based on the similarity between the query and documents via ranking models.In this paper,we explore a new paradigm of information retrieval without an explicit index but only with a pre-trained model.Instead,all of the knowledge of the documents is encoded into model parameters,which can be regarded as a differentiable indexer and optimized in an end-to-end manner.Specifically,we propose a pre-trained model-based information retrieval(IR)system called DynamicRetriever,which directly returns document identifiers for a given query.Under such a framework,we implement two variants to explore how to train the model from scratch and how to combine the advantages of dense retrieval models.Compared with existing search methods,the model-based IR system parameterizes the traditional static index with a pre-training model,which converts the document semantic mapping into a dynamic and updatable process.Extensive experiments conducted on the public search benchmark Microsoft machine reading comprehension(MS MARCO)verify the effectiveness and potential of our proposed new paradigm for information retrieval.