基于ML-BERT的海关商品编码层次分类方法

A Hierarchical Classification Method for Harmonization System Code Based on ML-BERT

本研究提出引入Multi-Low-Rank Adaption(Multi-LoRA)方法的语言模型ML-BERT对现有的层次编码分类识别任务进行了改进,通过对层次分类任务分别使用不同LoRA进行学习,按照编码构成顺序,将前置分类知识通过LoRA模块的计算注入到下层训练过程中。可以防止连续学习不同分类任务对预训练模型参数造成的影响,且针对不同层级的分类问题,不同LoRA模块可以捕获不同层次的文本特征。本文通过对试验参数进行控制变量分析,验证了模型引入多层LoRA的效果,在税号分类任务上获得了精度提升。

This study proposes a language model ML-BERT that introduces the Multi Low Rank Adaptation(Multi LoRA)method to improve the existing hierarchical coding classification and recognition tasks,by using different LoRA for learning hierarchical classification tasks,the pre classification knowledge is implicitly injected into the lower level training process through the calculation of the LoRA module according to the coding composition order.It can prevent continuous learning of different classification tasks from affecting the parameters of the pre training model,and for different levels of classification problems,different LoRA modules can capture different levels of text features.This article analyzes the control variables of experimental parameters,verified the effect of introducing multi-layer LoRA into the model,achieve accuracy improvement in tax number classification tasks.