Proposal of Japanese Vocabulary Difficulty Level Dictionaries for Automated Essay Scoring Support System Using Rubric

We are developing a Moodle plug-in,which is an AES(automated essay scoring)support system for the basic education of university students.Our system evaluates essays based on rubric,which has five evaluation viewpoints“Contents,Structure,Evidence,Style,and Skill”.Vocabulary level is one of the scoring items of Skill.It is calculated using Japanese Language Learners’Dictionaries constructed by Sunakawa et al.Since this does not fully cover the words used in the student-level essays,we found that there is a problem with the accuracy of the vocabulary level scoring.In this paper,we propose to construct comprehensive Japanese vocabulary difficulty level dictionaries using Japanese Wikipedia as the corpus.We apply Latent Dirichlet Allocation(LDA)to the Wikipedia corpus and find the word appearance probability as one of the indexes of word difficulty.We use the TF-IDF value instead of the LDA value of the words,which rarely appears.As a result,we constructed highly comprehensive Japanese vocabulary difficulty level dictionaries.We confirmed that the vocabulary level can be scored for all words in the test dataset by using the constructed dictionaries.

Automated Chinese Essay Scoring Based on Deep Learning

Writing is an important part of language learning and is considered the best approach to demonstrate the comprehensive language skills of students.Manually grading student essays is a time-consuming task;however,it is necessary.An automated essay scoring system can not only greatly improve the efficiency of essay scoring,but also provide more objective score.Therefore,many researchers have been exploring automated essay scoring techniques and tools.However,the technique of scoring Chinese essays is still limited,and its accuracy needs to be enhanced further.To improve the accuracy of the scoring model for a Chinese essay,we propose an automated scoring approach based on a deep learning model and validate its effect by conducting two comparison experiments.The experimental results indicate that the accuracy of the proposed model is significantly higher than that of multiple linear regression(MLR),which was commonly used in the past.The three accuracy rates of the proposed model are comparable to those of the novice teacher.The root mean square error(RMSE)of the proposed model is slightly lower than that of the novice teacher,and the correlation coefficient of the proposed model is also significantly higher than that of the novice teacher.Besides,when the predicted scores are not very low or very high,the two predicted models are as good as a novice teacher.However,when the predicted score is very high or very low,the results should be treated with caution.