Enhancement network: confining deep convolutional features based on hand-crafted rules

Despite convolutional neural network(CNN) is mature in many domains, the understanding of the directions where the parameters of the CNNs are learned towards, falls behind, and researches on the functions that the convolutional networks(ConvNets) learns are difficult to be explored. A method is proposed to guide ConvNets to learn towards the expected direction. First, for the sake of facilitating network converging, a novel feature enhancement framework, namely enhancement network(EN), is devised to learn parameters according to certain rules. Second, two types of hand-crafted rules, namely feature-sharpening(FS) and feature-amplifying(FA) are proposed to enable effective ENs, meanwhile are embedded into the CNN for the end-to-end learning. Specifically, the former is a tool sharpening convolutional features and the latter is the one amplifying convolutional features linearly. Both tools aim at the same spot achieving a stronger inductive bias and more straightforward loss functions. Finally, the experiments are conducted on the mixed National Institute of Standards and Technology(MNIST) and cooperative institute for Alaska research 10(CIFAR10) dataset. Experimental results demonstrate that ENs make a faster convergence by formulating hand-crafted rules.