一种有效的困难样本学习策略

Effective learning strategy for hard samples

针对困难样本在深度哈希算法中难以收敛以及过多的困难样本产生的噪声干扰问题,提出一种通过损失决定梯度的困难样本学习策略。首先,提出一种非均匀梯度归一化方法,通过计算困难与整体样本损失的比例,对整体样本反向传播梯度进行加权,提高模型对困难样本的学习能力;其次,针对存在过量困难样本的情况,设计了一种加权随机采样方法,根据损失大小对样本进行加权欠采样,滤除噪声并保留少量的困难样本避免过拟合。基于公开数据集,哈希特征检索平均精度值分别约提高了4.7%与3.4%。实验结果表明,该策略改进的哈希算法准确率优于对标哈希算法,能更好地学习到数据集中困难样本的特征信息。

To improve the learning efficiency for hard samples and reduce noise interference caused by superfluous hard samples in deep hash algorithm,a generic strategy called Loss to Gradient for hard sample learning is proposed.First,a non-uniform gradient normalization method is proposed to improve the learning ability of models for hard samples.Back propagation gradients are weighted by calculating the loss ratio between hard samples and all samples.Furthermore,a weighted random sampling method is designed for accuracy improvement with superfluous hard samples.According to the loss,training samples are weighted and under-sampled for noise filtering and a small number of hard samples are retained to avoid over-fitting.Based on open datasets,the average accuracy of hash feature retrieval is increased by 4.7%and 3.4%,respectively.Experimental results show that the improved method outperforms other benchmarking methods in accuracy,proving that the feature representation of hard samples in the dataset can be effectively learned.