基于混合maxout单元的卷积神经网络性能优化

Improving deep convolutional neural networks with mixed maxout units

针对深度卷积神经网络中maxout单元非最大特征无法传递、特征图像子空间池化表达能力不足的局限性,提出混合maxout(mixout,mixed maxout)单元。首先,计算相同输入在不同卷积变换下所形成的特征图像子空间的指数概率分布;其次,根据概率分布计算特征图像子空间的期望;最后,利用伯努利分布对子空间的最大值与期望值加权,均衡单元模型。分别构建基于mixout单元的简单模型和网中网模型进行实验,结果表明mixout单元模型性能较好。

The maxout units have the problem of not delivering non-max features,resulting in the insufficient of pooling operation over a subspace that is composed of several linear feature mappings,when they are applied in deep convolutional neural networks.The mixed maxout(mixout)units were proposed to deal with this constrain.Firstly,the exponential probability of the feature mappings getting from different linear transformations was computed.Then,the averaging of a subspace of different feature mappings by the exponential probability was computed.Finally,the output was randomly sampled from the max feature and the mean value by the Bernoulli distribution,leading to the better utilizing of model averaging ability of dropout.The simple models and network in network models was built to evaluate the performance of mixout units.The results show that mixout units based models have better performance.