针对传统自蒸馏方法存在数据预处理成本高、局部特征检测缺失,以及模型分类精度低的情况,提出了基于相似一致性的模型自蒸馏方法(Similarity and Consistency by Self-Distillation,SCD),提高模型分类精度。首先,对样本图像的不同层进...针对传统自蒸馏方法存在数据预处理成本高、局部特征检测缺失,以及模型分类精度低的情况,提出了基于相似一致性的模型自蒸馏方法(Similarity and Consistency by Self-Distillation,SCD),提高模型分类精度。首先,对样本图像的不同层进行学习得到特征图,通过特征权值分布获取注意力图。然后,计算Mini-batch内样本间注意力图的相似性获得相似一致性知识矩阵,构建基于相似一致性的知识,使得无须对实例数据进行失真处理或提取同一类别的数据来获取额外的实例间知识,避免了大量的数据预处理工作带来的训练成本高和训练复杂的问题。最后,将相似一致性知识矩阵在模型中间层之间单向传递,让浅层次的相似矩阵模仿深层次的相似矩阵,细化低层次的相似性,捕获更加丰富的上下文场景和局部特征,解决局部特征检测缺失问题,实现单阶段单向知识转移的自蒸馏。实验结果表明,采用基于相似一致性的模型自蒸馏方法:在公开数据集CIFAR100和TinyImageNet上,验证了SCD提取的相似一致性知识在模型自蒸馏中的有效性,相较于自注意力蒸馏方法(Self Attention Distillation,SAD)和保持相似性的知识蒸馏方法(Similarity-Preserving Knowledge Distillation,SPKD),分类精度平均提升1.42%;相较于基于深度监督的自蒸馏方法(Be Your Own Teacher,BYOT)和动态本地集成知识蒸馏方法(On-the-fly Native Ensemble,ONE),分类精度平均提升1.13%;相较于基于深度神经网络的数据失真引导自蒸馏方法(Data-Distortion Guided Self-Distillation,DDGSD)和基于类间的自蒸馏方法(Class-wise Self-Knowledge Distillation,CS-KD),分类精度平均提升1.23%。展开更多
The transport study of graphene based junctions has become one of the focuses in graphene research. There are two stacking configurations for monolayer–bilayer–monolayer graphene planar junctions. One is the two mon...The transport study of graphene based junctions has become one of the focuses in graphene research. There are two stacking configurations for monolayer–bilayer–monolayer graphene planar junctions. One is the two monolayer graphene contacting the same side of the bilayer graphene, and the other is the two-monolayer graphene contacting the different layers of the bilayer graphene. In this paper, according to the Landauer–Büttiker formula, we study the transport properties of these two configurations. The influences of the local gate potential in each part, the bias potential in bilayer graphene,the disorder and external magnetic field on conductance are obtained. We find the conductances of the two configurations can be manipulated by all of these effects. Especially, one can distinguish the two stacking configurations by introducing the bias potential into the bilayer graphene. The strong disorder and the external magnetic field will make the two stacking configurations indistinguishable in the transport experiment.展开更多
The Magnus Hall effect(MHE) is a new type of linear-response Hall effect, recently proposed to appear in two-dimensional(2D) nonmagnetic systems at zero magnetic field in the ballistic limit. The MHE arises from a sel...The Magnus Hall effect(MHE) is a new type of linear-response Hall effect, recently proposed to appear in two-dimensional(2D) nonmagnetic systems at zero magnetic field in the ballistic limit. The MHE arises from a self-rotating Bloch electron moving under a gradient-electrostatic potential, analogous to the Magnus effect in the macrocosm. Unfortunately, the MHE is usually accompanied by a trivial transverse signal, which hinders its experimental observation. We systematically investigate the material realization and experimental measurement of the MHE, based on symmetry analysis and first-principles calculations. It is found that both the out-ofplane mirror and in-plane two-fold symmetries can neutralize the trivial transverse signal to generate clean MHE signals. We choose two representative 2D materials, monolayer MoS_(2), and bilayer WTe_(2), to study the quantitative dependency of MHE signals on the direction of the electric field. The results are qualitatively consistent with the symmetry analysis, and suggest that an observable MHE signal requires giant Berry curvatures. Our results provide detailed guidance for the future experimental exploration of MHE.展开更多
文摘针对传统自蒸馏方法存在数据预处理成本高、局部特征检测缺失,以及模型分类精度低的情况,提出了基于相似一致性的模型自蒸馏方法(Similarity and Consistency by Self-Distillation,SCD),提高模型分类精度。首先,对样本图像的不同层进行学习得到特征图,通过特征权值分布获取注意力图。然后,计算Mini-batch内样本间注意力图的相似性获得相似一致性知识矩阵,构建基于相似一致性的知识,使得无须对实例数据进行失真处理或提取同一类别的数据来获取额外的实例间知识,避免了大量的数据预处理工作带来的训练成本高和训练复杂的问题。最后,将相似一致性知识矩阵在模型中间层之间单向传递,让浅层次的相似矩阵模仿深层次的相似矩阵,细化低层次的相似性,捕获更加丰富的上下文场景和局部特征,解决局部特征检测缺失问题,实现单阶段单向知识转移的自蒸馏。实验结果表明,采用基于相似一致性的模型自蒸馏方法:在公开数据集CIFAR100和TinyImageNet上,验证了SCD提取的相似一致性知识在模型自蒸馏中的有效性,相较于自注意力蒸馏方法(Self Attention Distillation,SAD)和保持相似性的知识蒸馏方法(Similarity-Preserving Knowledge Distillation,SPKD),分类精度平均提升1.42%;相较于基于深度监督的自蒸馏方法(Be Your Own Teacher,BYOT)和动态本地集成知识蒸馏方法(On-the-fly Native Ensemble,ONE),分类精度平均提升1.13%;相较于基于深度神经网络的数据失真引导自蒸馏方法(Data-Distortion Guided Self-Distillation,DDGSD)和基于类间的自蒸馏方法(Class-wise Self-Knowledge Distillation,CS-KD),分类精度平均提升1.23%。
基金supported by the National Natural Science Foundation of China(Grant No.11374219)the Jiangsu Provincial Natural Science Foundation,China(Grant No.BK20160007)the Research Fund for the Doctoral Program of Higher Education of China
文摘The transport study of graphene based junctions has become one of the focuses in graphene research. There are two stacking configurations for monolayer–bilayer–monolayer graphene planar junctions. One is the two monolayer graphene contacting the same side of the bilayer graphene, and the other is the two-monolayer graphene contacting the different layers of the bilayer graphene. In this paper, according to the Landauer–Büttiker formula, we study the transport properties of these two configurations. The influences of the local gate potential in each part, the bias potential in bilayer graphene,the disorder and external magnetic field on conductance are obtained. We find the conductances of the two configurations can be manipulated by all of these effects. Especially, one can distinguish the two stacking configurations by introducing the bias potential into the bilayer graphene. The strong disorder and the external magnetic field will make the two stacking configurations indistinguishable in the transport experiment.
基金Supported by the National Basic Research Program of China (Grant No.2019YFA0308403)the National Natural Science Foundation of China (Grant Nos.11822407,11947212,11704348,and NSFC20SC07)+1 种基金the China Postdoctoral Science Foundation (Grant No.2018M640513)the Hong Kong Research Grants Council (Grant Nos.26302118,16305019,and N HKUST626/18)。
文摘The Magnus Hall effect(MHE) is a new type of linear-response Hall effect, recently proposed to appear in two-dimensional(2D) nonmagnetic systems at zero magnetic field in the ballistic limit. The MHE arises from a self-rotating Bloch electron moving under a gradient-electrostatic potential, analogous to the Magnus effect in the macrocosm. Unfortunately, the MHE is usually accompanied by a trivial transverse signal, which hinders its experimental observation. We systematically investigate the material realization and experimental measurement of the MHE, based on symmetry analysis and first-principles calculations. It is found that both the out-ofplane mirror and in-plane two-fold symmetries can neutralize the trivial transverse signal to generate clean MHE signals. We choose two representative 2D materials, monolayer MoS_(2), and bilayer WTe_(2), to study the quantitative dependency of MHE signals on the direction of the electric field. The results are qualitatively consistent with the symmetry analysis, and suggest that an observable MHE signal requires giant Berry curvatures. Our results provide detailed guidance for the future experimental exploration of MHE.
