Piráis a reading comprehension dataset focused on the ocean,the Brazilian coast,and climate change,built from a collection of scientific abstracts and reports on these topics.This dataset represents a versatile l...Piráis a reading comprehension dataset focused on the ocean,the Brazilian coast,and climate change,built from a collection of scientific abstracts and reports on these topics.This dataset represents a versatile language resource,particularly useful for testing the ability of current machine learning models to acquire expert scientific knowledge.Despite its potential,a detailed set of baselines has not yet been developed for Pirá.By creating these baselines,researchers can more easily utilize Piráas a resource for testing machine learning models across a wide range of question answering tasks.In this paper,we define six benchmarks over the Pirádataset,covering closed generative question answering,machine reading comprehension,information retrieval,open question answering,answer triggering,and multiple choice question answering.As part of this effort,we have also produced a curated version of the original dataset,where we fixed a number of grammar issues,repetitions,and other shortcomings.Furthermore,the dataset has been extended in several new directions,so as to face the aforementioned benchmarks:translation of supporting texts from English into Portuguese,classification labels for answerability,automatic paraphrases of questions and answers,and multiple choice candidates.The results described in this paper provide several points of reference for researchers interested in exploring the challenges provided by the Pirádataset.展开更多
Long-document semantic measurement has great significance in many applications such as semantic searchs, plagiarism detection, and automatic technical surveys. However, research efforts have mainly focused on the sema...Long-document semantic measurement has great significance in many applications such as semantic searchs, plagiarism detection, and automatic technical surveys. However, research efforts have mainly focused on the semantic similarity of short texts. Document-level semantic measurement remains an open issue due to problems such as the omission of background knowledge and topic transition. In this paper, we propose a novel semantic matching method for long documents in the academic domain. To accurately represent the general meaning of an academic article, we construct a semantic profile in which key semantic elements such as the research purpose, methodology, and domain are included and enriched. As such, we can obtain the overall semantic similarity of two papers by computing the distance between their profiles. The distances between the concepts of two different semantic profiles are measured by word vectors. To improve the semantic representation quality of word vectors, we propose a joint word-embedding model for incorporating a domain-specific semantic relation constraint into the traditional context constraint. Our experimental results demonstrate that, in the measurement of document semantic similarity, our approach achieves substantial improvement over state-of-the-art methods, and our joint word-embedding model produces significantly better word representations than traditional word-embedding models.展开更多
基金The work was carried out at the Center for Artificial Intelligence(C4AI-USP)with support from the São Paulo Research Foundation(FAPESP grant#2019/07665-4)from the IBM Corporation.This research was also partially supported by ItaúUnibanco S.A.+1 种基金M.M.Joséand F.Nakasato have been supported by the ItaúScholarship Program(PBI)of the Data Science Center(C2D)of the Escola Politécnica da Universidade de São PauloWe acknowledge support by CAPES-Finance Code 001.A.H.R.Costa and F.G.Cozman were partially supported by CNPq grants 310085/2020-9 and 305753/2022-3 respectively.Paulo Pirozelli was supported by the FAPESP grant 2019/26762-0.
文摘Piráis a reading comprehension dataset focused on the ocean,the Brazilian coast,and climate change,built from a collection of scientific abstracts and reports on these topics.This dataset represents a versatile language resource,particularly useful for testing the ability of current machine learning models to acquire expert scientific knowledge.Despite its potential,a detailed set of baselines has not yet been developed for Pirá.By creating these baselines,researchers can more easily utilize Piráas a resource for testing machine learning models across a wide range of question answering tasks.In this paper,we define six benchmarks over the Pirádataset,covering closed generative question answering,machine reading comprehension,information retrieval,open question answering,answer triggering,and multiple choice question answering.As part of this effort,we have also produced a curated version of the original dataset,where we fixed a number of grammar issues,repetitions,and other shortcomings.Furthermore,the dataset has been extended in several new directions,so as to face the aforementioned benchmarks:translation of supporting texts from English into Portuguese,classification labels for answerability,automatic paraphrases of questions and answers,and multiple choice candidates.The results described in this paper provide several points of reference for researchers interested in exploring the challenges provided by the Pirádataset.
基金supported by the Foundation of the State Key Laboratory of Software Development Environment(No.SKLSDE-2015ZX-04)
文摘Long-document semantic measurement has great significance in many applications such as semantic searchs, plagiarism detection, and automatic technical surveys. However, research efforts have mainly focused on the semantic similarity of short texts. Document-level semantic measurement remains an open issue due to problems such as the omission of background knowledge and topic transition. In this paper, we propose a novel semantic matching method for long documents in the academic domain. To accurately represent the general meaning of an academic article, we construct a semantic profile in which key semantic elements such as the research purpose, methodology, and domain are included and enriched. As such, we can obtain the overall semantic similarity of two papers by computing the distance between their profiles. The distances between the concepts of two different semantic profiles are measured by word vectors. To improve the semantic representation quality of word vectors, we propose a joint word-embedding model for incorporating a domain-specific semantic relation constraint into the traditional context constraint. Our experimental results demonstrate that, in the measurement of document semantic similarity, our approach achieves substantial improvement over state-of-the-art methods, and our joint word-embedding model produces significantly better word representations than traditional word-embedding models.