大数据分析的无限深度神经网络方法

Big Data Analysis by Infinite Deep Neural Networks

深度神经网络(deep neural networks,DNNs)及其学习算法,作为成功的大数据分析方法,已为学术界和工业界所熟知.与传统方法相比,深度学习方法以数据驱动、能自动地从数据中提取特征(知识),对于分析非结构化、模式不明多变、跨领域的大数据具有显著优势.目前,在大数据分析中使用的深度神经网络主要是前馈神经网络(feedforward neural networks,FNNs),这种网络擅长提取静态数据的相关关系,适用于基于分类的数据应用场景.但是受到自身结构本质的限制,它提取数据时序特征的能力有限.无限深度神经网络(infinite deep neural networks)是一种具有反馈连接的回复式神经网络(recurrent neural networks,RNNs),本质上是一个动力学系统,网络状态随时间演化是这种网络的本质属性,它耦合了"时间参数",更加适用于提取数据的时序特征,从而进行大数据的预测.将这种网络的反馈结构在时间维度展开,随着时间的运行,这种网络可以"无限深",故称之为无限深度神经网络.重点介绍这种网络的拓扑结构和若干学习算法及其在语音识别和图像理解领域的成功实例.

Deep neural networks （DNNs） and their learning algorithms are well known in the academic community and industry as the most successful methods for big data analysis. Compared with traditional methods, deep learning methods use data-driven and can extract features （knowledge） automatically from data. Deep learning methods have significant advantages in analyzing unstructured, unknown and varied model and cross field big data. At present, the most widely used deep neural networks in big data analysis are feedforward neural networks （FNNs）. They work well in extracting the correlation from static data and suiting for data application scenarios based on classification. But limited by its intrinsic structure, the ability of feedforward neural networks to extract time sequence features is weak. Infinite deep neural networks, i.e. recurrent neural networks （RNNs） are dynamical systems essentially. Their essential character is that the states of the networks change with time and couple the time parameter. Hence they are very suit for extracting time sequence features. It means that infinite deep neural networks can perform the prediction of big data. If extending recurrent structure of recurrent neural networks in the time dimension, the depth of networks can be infinite with time running, so they are called infinite deep neural networks. In thispaper, we focus on the topology and some learning algorithms of intinite deep neural networks, and introduce some successful applications in speech recognition and image understanding.