Automated writing evaluation(1995-2022):Research trends and implications

Automated writing evaluation is the use of computer programs to evaluate essays written in an educational setting.This study aims to evaluate the state of the art and the trends of automated writing evaluation research.A total of 206 automated writing evaluation-related publications published in 88 journals between 1995 and 2022 were used as the data in our study.Bibliometric methods were employed to identify the publication trends,venues for publication,productive authors,highly cited articles,seminal works,and the most frequently explored topics across the past decades.The findings show that automated writing evaluation has become a fast-growing interdisciplinary area of interest.In addition,this line of research has witnessed a growing interest in topics pertaining to the latest technology,the perception of children,the teachers’feedback,and complex tasks.Implications of our study for future research are also discussed.

A Literature Review on Automated Writing Evaluation

This paper aims to make a brief review on automated writing evaluation(AWE) tools in English writing instruction both at home and abroad. In this paper, seven AWE tools are summarized, including Project Essay Grade(PEG), Intelligent Essay Assessor(IEA), the Electronic Essay Rater(E-rater) and Criterion, Intelli Metric and MY Access!, and Bayesian Essay Test Scoring System(BETSY), all of which are widely used abroad.While in China, i Write 2.0 and the Pigai system are used broadly.Through describing the developing process of AWE system, this paper tends to provide a general knowledge of AWE system.

Color and Shape efficiency for outlier detection from automated to user evaluation

The design of efficient representations is well established as a fruitful way to explore and analyze complex or large data.In these representations,data are encoded with various visual attributes depending on the needs of the representation itself.To make coherent design choices about visual attributes,the visual search field proposes guidelines based on the human brain’s perception of features.However,information visualization representations frequently need to depict more data than the amount these guidelines have been validated on.Since,the information visualization community has extended these guidelines to a wider parameter space.This paper contributes to this theme by extending visual search theories to an information visualization context.We consider a visual search task where subjects are asked to find an unknown outlier in a grid of randomly laid out distractors.Stimuli are defined by color and shape features for the purpose of visually encoding categorical data.The experimental protocol is made of a parameters space reduction step(i.e.,sub-sampling)based on a machine learning model,and a user evaluation to validate hypotheses and measure capacity limits.The results show that the major difficulty factor is the number of visual attributes that are used to encode the outlier.When redundantly encoded,the display heterogeneity has no effect on the task.When encoded with one attribute,the difficulty depends on that attribute heterogeneity until its capacity limit(7 for color,5 for shape)is reached.Finally,when encoded with two attributes simultaneously,performances drop drastically even with minor heterogeneity.

The Impact of Online Automated Writing Evaluation:A Case Study from Dalian

Automated writing evaluation （AWE） technology is being adopted in classrooms in China and the USA. This paper presents the results of a case study of the application of AWE in Dalian, China. The quasi-experimental study was conducted in 2010 in Chinese middle school English language classrooms. An effect size of 0.30 was found in favor of the experimental group using AWE as an online formative assessment. Student survey responses and teacher observations are presented as convergent evidence to illustrate the impact AWE technology has on teachers, students, and student achievement in English writing.

An Investigation on Chinese Elementary and Secondary School English Teachers' Perceptions on AWE as Formative Assessment Tools

The research is designed to examine current teaching practices and perceptions of elementary and secondary school language teachers so as to inform the design of a large-scale project on AWE （Automated Writing Evaluation） application in the Chinese classroom. The data indicated that teachers were confident in their computer use and understood formative assessment notions and their role in technology-enhanced teaching and learning. However, concerns were raised on the writing rubrics interpretation, the pedagogy on integrating AWE into teaching, and the suitability of the whole English software to Chinese learners. Solutions and suggestions were discussed. Sample lesson plans, pedagogy training and technical support were considered necessary for the participating teachers.