The analysis revealed that eight research studies ((Cheng, 2017); (Koltovskaia, 2020);(Li et al., 2017); (Liao, 2016); (Mohsen & Alshahrani, 2019); (Roscoe et al., 2017); (Wang, 2022); (Zhang, 2020)) focused on examining students’ perceptions of Automated Writing Evaluation (AWE) as both a scoring tool and a component of classroom instruction. The results of these studies indicated varied perceptions among students regarding AWE’s utility. Students’ perceptions were categorized into two main groups in this review: positive and negative perceptions. The Summary is presented on Table 2.

The findings indicate that a significant majority (70%) of the 138 students strongly agreed that Online OAF effectively identified strengths and weaknesses in their reflective journal writing. Moreover, more than half (55%) expressed their willingness to use OAF for providing feedback on their reflective writing, while only a small fraction (3%) showed reluctance towards future usage. The most favored feature of Automated Writing Evaluation (AWE) among students was the online automatic classification system, with 60% of respondents citing its usefulness. This system received praise for its ability to identify areas for improvement (40%), provide helpful suggestions and examples (23%), offer user-friendly interface (23%), and provide immediate feedback (13%). Overall, students appreciated the system's content, convenience, and speed. Additionally, a subgroup of students ( 12) believed that OAF could enhance their understanding of basic aspects of L2 writing. They highlighted two distinct advantages of OAF over teachers: its quick analysis and accessibility, as well as its capability for archiving and restoration. Despite some shortcomings, such as occasional inability to detect certain errors, students remained proactive in seeking solutions to their L2 writing challenges (Dikli & Bleyle, 2014)(Graham et al., 2015).

Further insights into positive perceptions were provided by (Li et al., 2017), who surveyed 135 students. The results showed that a small minority (3%) of the 31 students interviewed expressed high satisfaction with Criterion feedback, while the majority (77%) reported satisfaction, and 10% remained neutral. Among these students, 71% expressed satisfaction with Criterion’s grammar feedback, while 10% desired more detailed feedback. Grammar feedback was deemed the most helpful type, with 77% of students endorsing it. However, certain categories within grammar, such as run-on sentences (39%), possessives (19%), and prepositions (13%), posed challenges. Corrections were perceived as either easy (45%), difficult (10%), or variable depending on Criterion’s clarity (32%). Additionally, 29% of students claimed to have addressed all feedback, while over half (58%) admitted to ignoring some feedback. The majority (71%) of students reported a positive perception of AWE, citing its effectiveness in error identification (See Figure 2). Criterion was particularly instrumental in identifying errors related to articles (58%), wrong verb forms (19%), run-on sentences (19%), subject-verb agreement (13%), fragments (13%), wrong word forms (6%), pronouns (6%), possessives (3%), and faulty comparisons (3%). These findings corroborate previous research highlighting the core components of AWE, such as its scoring engine, which utilizes techniques like artificial intelligence and natural language processing (Stevenson, 2016). Moreover, the use of AWE to scaffold students’ writing ability has been shown to reduce grammatical errors in L2 writing (Liao, 2016), indicating that exposure to AWE feedback enhances learners’ understanding and application of English grammatical rules over time (Liao, 2016).

Additional positive perceptions were documented by Roscoe et al. (Roscoe et al., 2017), where students regarded Automated Writing Evaluation (AWE) as accurate, relevant, and useful in providing scoring and feedback. They expressed satisfaction with the quality of feedback provided by W-Pal, with 68.2% of 110 students expressing a willingness to use it again in the future. Similarly, Wang (Wang, 2022) reported that students exhibited a positive attitude towards Automated Essay Evaluation (AEE) systems, including iWrite, Awrite, and Pigai, rating them as greatly or moderately helpful. Most respondents appreciated the features of these systems, particularly regarding grammar, usage, mechanics, and syntactic complexity. While they were satisfied with the content analysis provided by AEE systems, they desired more feedback on discourse elements. Zhang's (Zhang, 2020) findings also highlighted students' positive perceptions of AWE feedback, noting its helpfulness in L2 writing, particularly in identifying collocation errors rarely addressed by teachers. The feedback was perceived as immediate and accurate, aiding students in revising their work and fostering an understanding of the importance of revision in the writing process. Moreover, it encouraged students to adopt the practice of multiple drafting when completing writing assignments outside of writing subjects. These results underscore the effectiveness of various AWE software types in enhancing the accuracy of scoring students' writing (Bridgeman et al., 2012)(Shermis, 2014).

To provide a balanced view, alongside the benefits of Automated Writing Evaluation (AWE), students’ perceptions reveal significant concerns. Table 3 outlines their dissatisfaction, including feedback quality, system comprehension of human language, scoring methods, and lack of explanatory reasons. Conversely, negative perceptions towards Automated Writing Evaluation (AWE) were highlighted by Koltovskaia’s (Koltovskaia, 2020) findings, which revealed that one out of two students believed AWE’s feedback to be less authoritative than that of teachers and possibly inaccurate. This skepticism arises from the belief that automated AWE systems lack the depth of understanding and contextual awareness that human teachers offer. Students might think that AWE tools are unable to appreciate the finer details of their writing or the specific context in which it was created, which can lead to doubts about the accuracy and reliability of the feedback. Furthermore, because AWE systems rely on algorithms and fixed criteria, there is concern that they may misinterpret intricate aspects of writing or not address the unique needs of individual students as effectively as feedback from a teacher. Consequently, AWE feedback was described as the most frustrating tool to read and comprehend by a reluctant student (Liao, 2016). The issues highlighted include the quality of feedback, the system’s comprehension of human language, scoring methods, and the lack of explanatory reasons. Among the surveyed students (52), dissatisfaction stemmed from various aspects, including the quality of feedback (35%), the system’s comprehension of human language (27%), scoring methods (19%), and lack of explanatory reasons (19%) (Cheng, 2017). While students expressed a keen interest in using AWE, particularly My Access, they felt the program, particularly its word bank functionality, did not adequately benefit them. They encountered difficulties in comprehending AWE feedback, particularly regarding content and organization, perceiving it as overly general and not tailored to their needs. In contrast, teacher feedback, especially regarding content and organization, was deemed clearer and more diagnostic than AWE feedback (Mohsen & Alshahrani, 2019). These findings echo previous research indicating AWE’s limitations in providing feedback that fully meets students’ needs (Stevenson & Phakiti, 2014).

The final objective of this study is to delineate the advantages and limitations of Automated Writing Evaluation (AWE) as both a scoring tool for students’ writing and a facilitator of writing instruction in the classroom. Numerous benefits have been identified, including enhanced linguistic accuracy, improved student performance, increased reliability and validity, effective scoring and feedback mechanisms, and both short-term and long-term impacts on student learning. These advantages have been validated by various studies, underscoring the promising potential of AWE for practical implementation in classroom settings (Table 4).

The results indicated that Automated Writing Evaluation (AWE) had an early effect on reducing the number of fragments and subject-verb disagreements in new texts, while the reduction of run-on sentences and ill-formed verbs became noticeable towards the end of the study phase. Despite variations among categories in both revisions and new texts, a consistent trend of linguistic growth facilitated by AWE was observed, leading to improved linguistic accuracy (Liao, 2016). AWE also demonstrated effectiveness in enhancing students' linguistic accuracy at a moderate level (57%) by addressing errors highlighted and suggested by the system, such as word form, articles, punctuation, spelling, prepositions, and spacing. Errors were visually highlighted in red, while suggestions were presented in green by the Automated Corrective Writing Feedback (ACWF) system (Koltovskaia, 2020). Moreover, AWE notably improved students' grammatical accuracy in both original and revised essays of the final task, underscoring its precision in identifying grammatical errors (Liao, 2016). WriteToLearn, a form of AWE, demonstrated greater consistency in rating papers compared to human raters, and was more accurate in identifying errors related to capitalization, spelling, punctuation, and connecting words, achieving precision rates of 100% for connecting word errors, 92.3% for capitalization errors, and 83.5% for recall. However, precision rates varied between 70% and 79% for errors related to subjectverb agreement, comma splices, and singular-plural nouns (Liu & Kunnan, 2016). (Ranalli et al., 2017) also reported a high accuracy rate of 70% in linguistic feature identification using Criterion. This is achievable because AWE systems employ algorithms designed to identify and rectify grammatical errors, spelling mistakes, and contextual issues, offering uniform feedback and personalized suggestions through sophisticated machine learning and natural language processing methods. These findings corroborate previous studies conducted by (Bridgeman et al., 2012)(Shermis, 2014), which emphasized the high accuracy of many AWE software types in scoring students' writing (Chapelle, 1999).

Some students exhibited improved performance following engagement with Automated Writing Evaluation (AWE) systems. This improvement was evidenced by higher revision scores, which correlated with positive changes in students' perceptions of feedback (Roscoe et al., 2017). Another study revealed that AWE had a substantial impact on students' revision (92.31%) and text modifications (97.44%), indicating both short-term and long-term effects on learning and performance (Link et al., 2022). AWE also fostered increased student engagement with learning tools and enhanced their habits of drafting and revising writing (Zhang, 2020). Additionally, Lachner et al. (Lachner et al., 2017) observed that students reported improvements in the global cohesion of their texts, attributed to the formative feedback provided by machine learning. (Mohsen & Alshahrani, 2019) suggested that AWE systems were valuable for evaluating students' writing and facilitating improvement. AWE improves students' writing ability by promptly identifying errors, offering instructional feedback, encouraging revisions, and tracking individual progress, all of which contribute to enhanced writing practices. While technology can serve as an assistant to instructors in second language learning, it cannot fully replace the role of instructors (Salaberry, 1999).

Ulmann’s (2019) research identified a comprehensive reflective writing model that demonstrated reliability and validity in detecting reflection in students’ writing. The evaluation model’s quality was theoretically deemed reliable and valid for detecting reflection. While reliability and validity tests utilized a rule-based approach across various model categories, empirical validation was achieved in only one category. Evaluation detection performance revealed that the machine learning component reliably differentiated between reflective and descriptive sentences and effectively distinguished categories of sentences with or without elements such as experience, feelings, personal beliefs, awareness of difficulties, perspective, lessons learned, and intention. Automated Writing Evaluation (AWE) systems have been positively perceived for their reliable scoring methods (Wang, 2022). (Mohsen & Alshahrani, 2019) revealed that My Access’s hybrid model enhanced writing accuracy through features like My Editor. These findings align with research by(Liu et al., 2018), who utilized a model emphasizing both technical and personalistic aspects across three phases (analysis, description, and critique). Notably, the model’s high reliability in inter-rater agreement underscores its effectiveness. Additionally, the results affirm previous findings indicating that AWE is more reliable and consistent than human raters in identifying writing errors ((Hutchison, 2007); (Shermis & Hamner, 2013)).

Giving Writing Feedback AWE is often considered more effective than human scoring and feedback due to its accessibility at any time and its capacity to provide detailed content feedback. Its integration into writing classrooms has proven effective in fostering students' learning autonomy, critical thinking, and overall writing proficiency. Additionally, AWE serves as an efficient tool for sharing learning resources (Wang, 2022). Students have shown preference for eight key characteristics of AEE, including its accessibility at any time, specificity, personalization, and comprehensibility compared to human rating systems (Wang, 2022). (Zhang & Hyland, 2018) also noted that 'AWE feedback offers discernible advantages over teacher feedback in terms of timeliness, convenience, multiple drafting opportunities, and even potential learner autonomy', corroborating previous findings by (Chen & Cheng, 2008) and (Dikli, 2006). The accessibility of feedback at any time and the ability to revise drafts multiple times align with earlier research (Cotos, 2015)(Stevenson & Phakiti, 2014)(Warschauer & Ware, 2006).

AWE has demonstrated both short-term and long-term impacts on learning and student performance, as indicated by (Link et al., 2022). Specifically, students who received instruction using AWE alongside teacher feedback exhibited long-term retention of accuracy improvement by fostering skill development, greater writing proficiency, and continuous improvement through progress tracking, whereas those taught solely with teacher feedback showed short-term retention (Link et al., 2022) by providing immediate feedback and encouraging prompt revisions. Additionally, AWE has shown a positive long-term effect on reducing instances of run-on sentences across all proficiency levels, as well as improvements in subject-verb agreement, with varying degrees of change observed between intermediate-high and advancedlow levels (Li et al., 2017)., whereas longterm, it contributes to skill growth, ongoing proficiency, and sustained development by monitoring progress over time. These longterm effects are attributed to sustained advancements in accuracy, allowing students to internalize knowledge gained from AWE corrective feedback and retain it in their long-term memory for future use (Bitchener, 2012).

The use of Automated Writing Evaluation (AWE) systems presents both advantages and limitations. Despite offering timely feedback, students often express skepticism due to the perceived disparity between AWE and human feedback. Studies show AWE's weaknesses in comparison to human raters, raising questions about its effectiveness in facilitating meaningful revisions and its impact on students' writing development.

AWE also presents certain limitations, such as students’ skepticism towards the feedback provided (Table 5). Skepticism often arises from the perceived gap between teacher feedback, which is seen as more influential due to its expert knowledge, personalized advice, and interactive nature, and AWE feedback, which lacks these strengths and is criticized for its inability to provide nuanced, authoritative guidance and to prompt thorough revisions (Koltovskaia, 2020). Particularly, lower-level students have expressed frustration with receiving scores and feedback devoid of human interaction (Liao, 2016). Additionally, students have shown a tendency to prioritize the quantity of feedback over the quality of revision suggestions. While the use of causal verbs decreased from the initial to the final draft, there was no corresponding increase in the use of causal nouns. Furthermore, the findings of Saricaoglu’s (Saricaoglu, 2019) research study did not indicate any long-term effects.

AWE also exhibits several weaknesses, notably in comparison to human raters. In a study by Liu and Kunnan (Liu & Kunnan, 2016), human raters outperformed AWE in rating students’ writing, assessing 326 essays compared to AWE’s 319 essays. AWE missed identifying 7 essays, and its accuracy was lower than that of human raters, detecting only 15 errors compared to the 22 identified by human raters. Additionally, studies such as iWrite (Qian et al., 2020) reported no correlations between AWE and human scores, indicating poor automated scoring quality. AWE consistently yielded lower scores than human raters, and there was no correlation between the presented scores and feedback quality. These findings were echoed by Roscoe et al. (Roscoe et al., 2017), highlighting that the scoring accuracy did not align with the initial expectations of AWE quality. Furthermore, Prompt-specific AWE engines were found to be limited in their applicability, as they could only evaluate texts written in response to pre-trained prompts (Stevenson & Phakiti, 2019).

The research findings revealed that automated feedback primarily provided low-level feedback, with lexical meaning receiving a 66.2% rating and a mere 5.7% uptake rate, while grammar and mechanics garnered a 68.7% rating with a 46.4% uptake rate. When compared to peer and teacher feedback, automated feedback exhibited the lowest uptake rate, followed by peer feedback, with teacher feedback being the most authoritative among the three (Tian & Zhou, 2020). These findings corroborate previous research highlighting the detrimental effects of AWE when acting as a non-human audience (Stevenson, 2016).