AI-infused Semantic Model to Enrich and Expand Programming Question Generation

Creating practice questions for programming learning is not easy.It requires the instructor to diligently organize heterogeneous learning resources,that is,conceptual programming concepts and procedural programming rules.Today’s programming question generation(PQG)is still largely relying on the demanding creation task performed by the instructors without advanced technological support.In this work,we propose a semantic PQG model that aims to help the instructor generate new programming questions and expand the assessment items.The PQG model is designed to transform conceptual and procedural programming knowledge from textbooks into a semantic network by the Local Knowledge Graph(LKG)and Abstract Syntax Tree(AST).For any given question,the model queries the established network to find related code examples and generates a set of questions by the associated LKG/AST semantic structures.We conduct analysis to compare instructor-made questions from 9 undergraduate introductory programming courses and textbook questions.The results show that the instructormade questions had much simpler complexity than the textbook ones.The disparity of topic distribution intrigued us to further research the breadth and depth of question quality and also to investigate the complexity of the questions in relation to the student performances.Finally,we report a user study results on the proposed Artificial Intelligent-infused semantic PQG model in examining the machine-generated questions’quality.

Leveraging Structured Information from a Passage to Generate Questions

Question Generation(QG)is the task of utilizing Artificial Intelligence(AI)technology to generate questions that can be answered by a span of text within a given passage.Existing research on QG in the educational field struggles with two challenges:the mainstream QG models based on seq-to-seq fail to utilize the structured information from the passage;the other is the lack of specialized educational QG datasets.To address the challenges,a specialized QG dataset,reading comprehension dataset from examinations for QG(named RACE4QG),is reconstructed by applying a new answer tagging approach and a data-filtering strategy to the RACE dataset.Further,an end-to-end QG model,which can exploit the intra-and inter-sentence information to generate better questions,is proposed.In our model,the encoder utilizes a Gated Recurrent Units(GRU)network,which takes the concatenation of word embedding,answer tagging,and Graph Attention neTworks(GAT)embedding as input.The hidden states of the GRU are operated with a gated self-attention to obtain the final passage-answer representation,which will be fed to the decoder.Results show that our model outperforms baselines on automatic metrics and human evaluation.Consequently,the model improves the baseline by 0.44,1.32,and 1.34 on BLEU-4,ROUGE-L,and METEOR metrics,respectively,indicating the effectivity and reliability of our model.Its gap with human expectations also reflects the research potential.