Recently,pre-trained language representation models such as bidirec-tional encoder representations from transformers(BERT)have been performing well in commonsense question answering(CSQA).However,there is a problem th...Recently,pre-trained language representation models such as bidirec-tional encoder representations from transformers(BERT)have been performing well in commonsense question answering(CSQA).However,there is a problem that the models do not directly use explicit information of knowledge sources existing outside.To augment this,additional methods such as knowledge-aware graph network(KagNet)and multi-hop graph relation network(MHGRN)have been proposed.In this study,we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers(ALBERT)with knowledge graph information extraction technique.We also propose to applying the novel method,schema graph expansion to recent language models.Then,we analyze the effect of applying knowledge graph-based knowledge extraction techniques to recent pre-trained language models and confirm that schema graph expansion is effective in some extent.Furthermore,we show that our proposed model can achieve better performance than existing KagNet and MHGRN models in CommonsenseQA dataset.展开更多
Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neu...Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neural networks to model inference.However,traditional knowledge graph are mostly concept-based,ignoring direct path evidence necessary for accurate reasoning.In this paper,we propose MRGNN(Meta-path Reasoning Graph Neural Network),a novel model that comprehensively captures sequential semantic information from concepts and paths.In MRGNN,meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously.We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets,showing the effectiveness of MRGNN.Also,we conduct further ablation experiments and explain the reasoning behavior through the case study.展开更多
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea Government(MSIT)(No.2020R1G1A1100493).
文摘Recently,pre-trained language representation models such as bidirec-tional encoder representations from transformers(BERT)have been performing well in commonsense question answering(CSQA).However,there is a problem that the models do not directly use explicit information of knowledge sources existing outside.To augment this,additional methods such as knowledge-aware graph network(KagNet)and multi-hop graph relation network(MHGRN)have been proposed.In this study,we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers(ALBERT)with knowledge graph information extraction technique.We also propose to applying the novel method,schema graph expansion to recent language models.Then,we analyze the effect of applying knowledge graph-based knowledge extraction techniques to recent pre-trained language models and confirm that schema graph expansion is effective in some extent.Furthermore,we show that our proposed model can achieve better performance than existing KagNet and MHGRN models in CommonsenseQA dataset.
基金supported by the Key Research and Development Program of Hubei Province(2020BAB017)the Scientific Research Center Program of National Language Commission(ZDI135-135)the Fundamental Research Funds for the Central Universities(KJ02502022-0155,CCNU22XJ037).
文摘Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neural networks to model inference.However,traditional knowledge graph are mostly concept-based,ignoring direct path evidence necessary for accurate reasoning.In this paper,we propose MRGNN(Meta-path Reasoning Graph Neural Network),a novel model that comprehensively captures sequential semantic information from concepts and paths.In MRGNN,meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously.We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets,showing the effectiveness of MRGNN.Also,we conduct further ablation experiments and explain the reasoning behavior through the case study.
