A Theoretically Grounded Question Answering Data Set for Evaluating Machine Common Sense

Achieving machine common sense has been a longstanding problem within Artificial Intelligence.Thus far,benchmark data sets that are grounded in a theory of common sense and can be used to conduct rigorous,semantic evaluations of common sense reasoning(CSR)systems have been lacking.One expectation of the AI community is that neuro-symbolic reasoners can help bridge this gap towards more dependable systems with common sense.We propose a novel benchmark,called Theoretically Grounded common sense Reasoning(TG-CSR),modeled as a set of question answering instances,with each instance grounded in a semantic category of common sense,such as space,time,and emotions.The benchmark is few-shot i.e.,only a few training and validation examples are provided in the public release to avoid the possibility of overfitting.Results from recent evaluations suggest that TG-CSR is challenging even for state-of-the-art statistical models.Due to its semantic rigor,this benchmark can be used to evaluate the common sense reasoning capabilities of neuro-symbolic systems.

The Semantic Data Dictionary-An Approach for Describing and Annotating Data

It is common practice for data providers to include text descriptions for each column when publishing data sets in the form of data dictionaries.While these documents are useful in helping an end-user properly interpret the meaning of a column in a data set,existing data dictionaries typically are not machine-readable and do not follow a common specification standard.We introduce the Semantic Data Dictionary,a specification that formalizes the assignment of a semantic representation of data,enabling standardization and harmonization across diverse data sets.In this paper,we present our Semantic Data Dictionary work in the context of our work with biomedical data;however,the approach can and has been used in a wide range of domains.The rendition of data in this form helps promote improved discovery,interoperability,reuse,traceability,and reproducibility.We present the associated research and describe how the Semantic Data Dictionary can help address existing limitations in the related literature.We discuss our approach,present an example by annotating portions of the publicly available National Health and Nutrition Examination Survey data set,present modeling challenges,and describe the use of this approach in sponsored research,including our work on a large National Institutes of Health(NIH)-funded exposure and health data portal and in the RPI-IBM collaborative Health Empowerment by Analytics,Learning,and Semantics project.