Dynamic networks have become popular to enhance the model capacity while maintaining efficient inference by dynamically generating the weight based on over-parameters.They bring much more parameters and increase the d...Dynamic networks have become popular to enhance the model capacity while maintaining efficient inference by dynamically generating the weight based on over-parameters.They bring much more parameters and increase the difficulty of the training.In this paper,a multi-layer dynamic convolution(MDConv) is proposed,which scatters the over-parameters over multi-layers with fewer parameters but stronger model capacity compared with scattering horizontally;it uses the expanding form where the attention is applied to the features to facilitate the training;it uses the compact form where the attention is applied to the weights to maintain efficient inference.Moreover,a multi-layer asymmetric convolution(MAConv) is proposed,which has no extra parameters and computation cost at inference time compared with static convolution.Experimental results show that MDConv achieves better accuracy with fewer parameters and significantly facilitates the training;MAConv enhances the accuracy without any extra cost of storage or computation at inference time compared with static convolution.展开更多
基金Supported by the National Key Research and Development Program of China(No.2016YFB1000601)the Standardization Pilot Research Project of Chinese Academy of Sciences(No.20194620)。
文摘Dynamic networks have become popular to enhance the model capacity while maintaining efficient inference by dynamically generating the weight based on over-parameters.They bring much more parameters and increase the difficulty of the training.In this paper,a multi-layer dynamic convolution(MDConv) is proposed,which scatters the over-parameters over multi-layers with fewer parameters but stronger model capacity compared with scattering horizontally;it uses the expanding form where the attention is applied to the features to facilitate the training;it uses the compact form where the attention is applied to the weights to maintain efficient inference.Moreover,a multi-layer asymmetric convolution(MAConv) is proposed,which has no extra parameters and computation cost at inference time compared with static convolution.Experimental results show that MDConv achieves better accuracy with fewer parameters and significantly facilitates the training;MAConv enhances the accuracy without any extra cost of storage or computation at inference time compared with static convolution.
