Data Augmentation Technology Driven By Image Style Transfer in Self-Driving Car Based on End-to-End Learning

With the advent of deep learning,self-driving schemes based on deep learning are becoming more and more popular.Robust perception-action models should learn from data with different scenarios and real behaviors,while current end-to-end model learning is generally limited to training of massive data,innovation of deep network architecture,and learning in-situ model in a simulation environment.Therefore,we introduce a new image style transfer method into data augmentation,and improve the diversity of limited data by changing the texture,contrast ratio and color of the image,and then it is extended to the scenarios that the model has been unobserved before.Inspired by rapid style transfer and artistic style neural algorithms,we propose an arbitrary style generation network architecture,including style transfer network,style learning network,style loss network and multivariate Gaussian distribution function.The style embedding vector is randomly sampled from the multivariate Gaussian distribution and linearly interpolated with the embedded vector predicted by the input image on the style learning network,which provides a set of normalization constants for the style transfer network,and finally realizes the diversity of the image style.In order to verify the effectiveness of the method,image classification and simulation experiments were performed separately.Finally,we built a small-sized smart car experiment platform,and apply the data augmentation technology based on image style transfer drive to the experiment of automatic driving for the first time.The experimental results show that:(1)The proposed scheme can improve the prediction accuracy of the end-to-end model and reduce the model’s error accumulation;(2)the method based on image style transfer provides a new scheme for data augmentation technology,and also provides a solution for the high cost that many deep models rely heavily on a large number of label data.

An End-to-End Machine Learning Framework for Predicting Common Geriatric Diseases

Interdisciplinary applications between information technology and geriatrics have been accelerated in recent years by the advancement of artificial intelligence,cloud computing,and 5G technology,among others.Meanwhile,applications developed by using the above technologies make it possible to predict the risk of age-related diseases early,which can give caregivers time to intervene and reduce the risk,potentially improving the health span of the elderly.However,the popularity of these applications is still limited for several reasons.For example,many older people are unable or unwilling to use mobile applications or devices(e.g.smartphones)because they are relatively complex operations or time-consuming for older people.In this work,we design and implement an end-to-end framework and integrate it with the WeChat platform to make it easily accessible to elders.In this work,multifactorial geriatric assessment data can be collected.Then,stacked machine learning models are trained to assess and predict the incidence of common diseases in the elderly.Experimental results show that our framework can not only provide more accurate prediction(precision:0.8713,recall:0.8212)for several common elderly diseases,but also very low timeconsuming(28.6 s)within a workflow compared to some existing similar applications.

Generative Adversarial Network-Based Electromagnetic Signal Classification: A Semi- Supervised Learning Framework

Generative adversarial network(GAN)has achieved great success in many fields such as computer vision,speech processing,and natural language processing,because of its powerful capabilities for generating realistic samples.In this paper,we introduce GAN into the field of electromagnetic signal classification(ESC).ESC plays an important role in both military and civilian domains.However,in many specific scenarios,we can’t obtain enough labeled data,which cause failure of deep learning methods because they are easy to fall into over-fitting.Fortunately,semi-supervised learning(SSL)can leverage the large amount of unlabeled data to enhance the classification performance of classifiers,especially in scenarios with limited amount of labeled data.We present an SSL framework by incorporating GAN,which can directly process the raw in-phase and quadrature(IQ)signal data.According to the characteristics of the electromagnetic signal,we propose a weighted loss function,leading to an effective classifier to realize the end-to-end classification of the electromagnetic signal.We validate the proposed method on both public RML2016.04c dataset and real-world Aircraft Communications Addressing and Reporting System(ACARS)signal dataset.Extensive experimental results show that the proposed framework obtains a significant increase in classification accuracy compared with the state-of-the-art studies.

An Overview of Wireless Communication Technology Using Deep Learning

with the development of 5G,the future wireless communication network tends to be more and more intelligent.In the face of new service de-mands of communication in the future such as super-heterogeneous network,multiple communication sce-narios,large number of antenna elements and large bandwidth,new theories and technologies of intelli-gent communication have been widely studied,among which Deep Learning(DL)is a powerful technology in artificial intelligence(AI).It can be trained to con-tinuously learn to update the optimal parameters.This paper reviews the latest research progress of DL in in-telligent communication,and emphatically introduces five scenarios including Cognitive Radio(CR),Edge Computing(EC),Channel Measurement(CM),End to end Encoder/Decoder(EED)and Visible Light Com-munication(VLC).The prospect and challenges of further research and development in the future are also discussed.

An Overview and Experimental Study of Learning-Based Optimization Algorithms for the Vehicle Routing Problem

The vehicle routing problem(VRP)is a typical discrete combinatorial optimization problem,and many models and algorithms have been proposed to solve the VRP and its variants.Although existing approaches have contributed significantly to the development of this field,these approaches either are limited in problem size or need manual intervention in choosing parameters.To solve these difficulties,many studies have considered learning-based optimization(LBO)algorithms to solve the VRP.This paper reviews recent advances in this field and divides relevant approaches into end-to-end approaches and step-by-step approaches.We performed a statistical analysis of the reviewed articles from various aspects and designed three experiments to evaluate the performance of four representative LBO algorithms.Finally,we conclude the applicable types of problems for different LBO algorithms and suggest directions in which researchers can improve LBO algorithms.

Building Semantic Communication System via Molecules:An End-to-End Training Approach

The concept of semantic communication provides a novel approach for applications in scenarios with limited communication resources.In this paper,we propose an end-to-end(E2E)semantic molecular communication system,aiming to enhance the efficiency of molecular communication systems by reducing the transmitted information.Specifically,following the joint source channel coding paradigm,the network is designed to encode the task-relevant information into the concentration of the information molecules,which is robust to the degradation of the molecular communication channel.Furthermore,we propose a channel network to enable the E2E learning over the non-differentiable molecular channel.Experimental results demonstrate the superior performance of the semantic molecular communication system over the conventional methods in classification tasks.

An autonomic joint radio resource management algorithm in end-to-end reconfigurable system

This paper presents the multi-step Q-learning(MQL)algorithm as an autonomic approach to thejoint radio resource management(JRRM)among heterogeneous radio access technologies(RATs)in theB3G environment.Through the'trial-and-error'on-line learning process,the JRRM controller can con-verge to the optimized admission control policy.The JRRM controller learns to give the best allocation foreach session in terms of both the access RAT and the service bandwidth.Simulation results show that theproposed algorithm realizes the autonomy of JRRM and achieves well trade-off between the spectrum utilityand the blocking probability comparing to the load-balancing algorithm and the utility-maximizing algo-rithm.Besides,the proposed algorithm has better online performances and convergence speed than theone-step Q-learning(QL)algorithm.Therefore,the user statisfaction degree could be improved also.

End-to-End Auto-Encoder System for Deep Residual Shrinkage Network for AWGN Channels

With the rapid development of deep learning methods, the data-driven approach has shown powerful advantages over the model-driven one. In this paper, we propose an end-to-end autoencoder communication system based on Deep Residual Shrinkage Networks (DRSNs), where neural networks (DNNs) are used to implement the coding, decoding, modulation and demodulation functions of the communication system. Our proposed autoencoder communication system can better reduce the signal noise by adding an “attention mechanism” and “soft thresholding” modules and has better performance at various signal-to-noise ratios (SNR). Also, we have shown through comparative experiments that the system can operate at moderate block lengths and support different throughputs. It has been shown to work efficiently in the AWGN channel. Simulation results show that our model has a higher Bit-Error-Rate (BER) gain and greatly improved decoding performance compared to conventional modulation and classical autoencoder systems at various signal-to-noise ratios.

基于端到端学习机制的高速公路行驶轨迹曲率预测

本文中基于端到端学习机制进行了高速公路场景下的车辆行驶轨迹预测。首先,为量化表达行驶轨迹,并对预测结果进行合理评价,建立了行车轨迹曲率预测模型与评价体系。然后,针对端到端的行驶轨迹曲率预测训练集中驾驶员行为决策的不确定性导致性能不佳的问题,采用场景切分和特征预提取的方法进行优化和实车试验验证。结果表明,该方法提高了高速公路行驶轨迹预测的准确性和可靠性。

Applying machine learning to market analysis:Knowing your luxury consumer

Chinese consumer research in the luxury sector is the emphasis in the business research field.However,it can be cost-intensive or time-consuming to interpret big data from any research conducted in the field.In this paper,the researchers created a machine-learning model to help minimize those research barriers.This study analyzed Chinese luxury consumption behavior,while the Chinese contributed 33%of the global luxury market in 2018 and play as a growth engine in the luxury market(Bain&Company.2019.https://www.bain.com/insights/whats-powering-chinas-market-for-luxury-goods/).The researchers interpreted this analysis using machine-learning algorithms through different sets of conditions and then proposed an understandable and highly accurate machine-learning model.Unlike traditional statistical methods,which rely on domain experts to create handcrafted features,this paper proposes an unsupervised end-to-end model that can directly and accurately process questionnaire data without human intervention.This paper also demonstrates how to practically apply an automatic unsupervised analysis method(PCA)to find inferences in the big data,and helps interpret the implied meaning to the questions.

Fast Mesh Reconstruction from Single View Based on GCN and Topology Modification

3D reconstruction based on single view aims to reconstruct the entire 3D shape of an object from one perspective.When existing methods reconstruct the mesh surface of complex objects,the surface details are difficult to predict and the reconstruction visual effect is poor because the mesh representation is not easily integrated into the deep learning framework;the 3D topology is easily limited by predefined templates and inflexible,and unnecessary mesh self-intersections and connections will be generated when reconstructing complex topology,thus destroying the surface details;the training of the reconstruction network is limited by the large amount of information attached to the mesh vertices,and the training time of the reconstructed network is too long.In this paper,we propose a method for fast mesh reconstruction from single view based on Graph Convolutional Network(GCN)and topology modification.We use GCN to ensure the generation of high-quality mesh surfaces and use topology modification to improve the flexibility of the topology.Meanwhile,a feature fusion method is proposed to make full use of the features of each stage of the image hierarchically.We use 3D open dataset ShapeNet to train our network and add a new weight parameter to speed up the training process.Extensive experiments demonstrate that our method can not only reconstruct object meshes on complex topological surfaces,but also has better qualitative and quantitative results.

End-to-end data-driven modeling framework for automated and trustworthy short-term building energy load forecasting

Conventional automated machine learning(AutoML)technologies fall short in preprocessing low-quality raw data and adapting to varying indoor and outdoor environments,leading to accuracy reduction in forecasting short-term building energy loads.Moreover,their predictions are not transparent because of their black box nature.Hence,the building field currently lacks an AutoML framework capable of data quality enhancement,environment self-adaptation,and model interpretation.To address this research gap,an improved AutoML-based end-to-end data-driven modeling framework is proposed.Bayesian optimization is applied by this framework to find an optimal data preprocessing process for quality improvement of raw data.It bridges the gap where conventional AutoML technologies cannot automatically handle missing data and outliers.A sliding window-based model retraining strategy is utilized to achieve environment self-adaptation,contributing to the accuracy enhancement of AutoML technologies.Moreover,a local interpretable model-agnostic explanations-based approach is developed to interpret predictions made by the improved framework.It overcomes the poor interpretability of conventional AutoML technologies.The performance of the improved framework in forecasting one-hour ahead cooling loads is evaluated using two-year operational data from a real building.It is discovered that the accuracy of the improved framework increases by 4.24%–8.79%compared with four conventional frameworks for buildings with not only high-quality but also low-quality operational data.Furthermore,it is demonstrated that the developed model interpretation approach can effectively explain the predictions of the improved framework.The improved framework offers a novel perspective on creating accurate and reliable AutoML frameworks tailored to building energy load prediction tasks and other similar tasks.

End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

Recent years have seen the rapid development of autonomous driving systems,which are typically designed in a hierarchical architecture or an end-to-end architecture.The hierarchical architecture is always complicated and hard to design,while the end-to-end architecture is more promising due to its simple structure.This paper puts forward an end-to-end autonomous driving method through a deep reinforcement learning algorithm Dueling Double Deep Q-Network,making it possible for the vehicle to learn end-to-end driving by itself.This paper firstly proposes an architecture for the end-to-end lane-keeping task.Unlike the traditional image-only state space,the presented state space is composed of both camera images and vehicle motion information.Then corresponding dueling neural network structure is introduced,which reduces the variance and improves sampling efficiency.Thirdly,the proposed method is applied to The Open Racing Car Simulator(TORCS)to demonstrate its great performance,where it surpasses human drivers.Finally,the saliency map of the neural network is visualized,which indicates the trained network drives by observing the lane lines.A video for the presented work is available online,https://youtu.be/76ciJ mIHMD8 or https://v.youku.com/v_show/id_XNDM4 ODc0M TM4NA==.html.

End-to-end optimization of a diffractive optical element and aberration correction for integral imaging

In the integral imaging light field display, the introduction of a diffractive optical element (DOE) can solve the problem of limited depth of field of the traditional lens. However, the strong aberration of the DOE significantly reduces the final display quality. Thus, herein, an end-to-end joint optimization method for optimizing DOE and aberration correction is proposed. The DOE model is established using thickness as the variable, and a deep learning network is built to preprocess the composite image loaded on the display panel. The simulation results show that the peak signal to noise ratio value of the optimized image increases by 8 dB, which confirms that the end-to-end joint optimization method can effectively reduce the aberration problem.

Unsupervised Dialogue State Tracking for End-to-End Task-Oriented Dialogue with a Multi-Span Prediction Network

This paper focuses on end-to-end task-oriented dialogue systems,which jointly handle dialogue state tracking(DST)and response generation.Traditional methods usually adopt a supervised paradigm to learn DST from a manually labeled corpus.However,the annotation of the corpus is costly,time-consuming,and cannot cover a wide range of domains in the real world.To solve this problem,we propose a multi-span prediction network(MSPN)that performs unsupervised DST for end-to-end task-oriented dialogue.Specifically,MSPN contains a novel split-merge copy mechanism that captures long-term dependencies in dialogues to automatically extract multiple text spans as keywords.Based on these keywords,MSPN uses a semantic distance based clustering approach to obtain the values of each slot.In addition,we propose an ontology-based reinforcement learning approach,which employs the values of each slot to train MSPN to generate relevant values.Experimental results on single-domain and multi-domain task-oriented dialogue datasets show that MSPN achieves state-of-the-art performance with significant improvements.Besides,we construct a new Chinese dialogue dataset MeDial in the low-resource medical domain,which further demonstrates the adaptability of MSPN.

A novel method for atomization energy prediction based on natural-parameter network

Atomization energy(AE)is an important indicator for measuring material stability and reactivity,which refers to the energy change when a polyatomic molecule decomposes into its constituent atoms.Predicting AE based on the structural information of molecules has been a focus of researchers,but existing methods have limitations such as being time-consuming or requiring complex preprocessing and large amounts of training data.Deep learning(DL),a new branch of machine learning(ML),has shown promise in learning internal rules and hierarchical representations of sample data,making it a potential solution for AE prediction.To address this problem,we propose a natural-parameter network(NPN)approach for AE prediction.This method establishes a clearer statistical interpretation of the relationship between the network’s output and the given data.We use the Coulomb matrix(CM)method to represent each compound as a structural information matrix.Furthermore,we also designed an end-to-end predictive model.Experimental results demonstrate that our method achieves excellent performance on the QM7 and BC2P datasets,and the mean absolute error(MAE)obtained on the QM7 test set ranges from 0.2 kcal/mol to 3 kcal/mol.The optimal result of our method is approximately an order of magnitude higher than the accuracy of 3 kcal/mol in published works.Additionally,our approach significantly accelerates the prediction time.Overall,this study presents a promising approach to accelerate the process of predicting structures using DL,and provides a valuable contribution to the field of chemical energy prediction.

MILI:Multi-person inference from a low-resolution image

Existing multi-person reconstruction methods require the human bodies in the input image to occupy a considerable portion of the picture.However,low-resolution human objects are ubiquitous due to trade-offbetween the field of view and target distance given a limited camera resolution.In this paper,we propose an end-to-end multi-task framework for multi-person inference from a low-resolution image(MILI).To perceive more information from a low-resolution image,we use pair-wise images at high resolution and low resolution for training,and design a restoration network with a simple loss for better feature extraction from the low-resolution image.To address the occlusion problem in multi-person scenes,we propose an occlusion-aware mask prediction network to estimate the mask of each person during 3D mesh regression.Experimental results on both small-scale scenes and large-scale scenes demonstrate that our method outperforms the state-of-the-art methods both quantitatively and qualitatively.The code is available at http://cic.tju.edu.cn/faculty/likun/projects/MILI.