A Comparative Analysis of AI-Powered and Teacher-Led Feedback: Investigating Student Perceptions and Writing Performance
Keywords:
Artificial Intelligence, ESL Writing, Assessment, Automated EvaluationAbstract
This study examines the comparative effectiveness of AI-powered and teacher-led feedback on the academic writing development of first-semester postgraduate ESL students. The study employs a mixed-methods research design to address the demand for efficient, personalised feedback in large classrooms. Utilising the Write & Improve (Cambridge) platform, AI-generated feedback was provided to one group, while another received teacher feedback, with students blind to the source. Quantitative data from pre- and post-tests assessed improvements in grammar, vocabulary, organization, and coherence, while qualitative data from surveys and semi-structured interviews captured perceptions of feedback usefulness. Human rater evaluations were conducted to compare AI and teacher feedback alignment. Preliminary findings indicate that AI-powered feedback is comparable to teacher-led feedback in several writing aspects, though issues like accuracy and ethical transparency require further exploration. The research highlights AI's potential to complement traditional feedback, advocating for a balanced approach to enhance ESL learning outcomes.
References
Bitchener, J., & Ferris, D. (2012). Written corrective feedback in second language acquisition and writing. Routledge.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.
Canagarajah, A. S. (2002). A geopolitics of academic writing. University of Pittsburgh Press.
Chen, Y., Li, J., & Warschauer, M. (2021). AI in language learning: Research, practice, and ethics. Language Learning & Technology, 25(3), 102-120.
Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). Routledge.
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge University Press.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). SAGE Publications.
Dikli, S. (2006). An overview of automated scoring of essays. Journal of Technology, Learning, and Assessment, 5(1).
Dörnyei, Z. (2007). Research methods in applied linguistics. Oxford University Press.
Ferris, D. (2003). Response to student writing: Implications for second-language students. Routledge.
Guo, Q., & Zhang, L. J. (2020). Optimizing EFL writing instruction through automated writing evaluation: Students’ perceptions of Criterion in a writing for academic purposes course. Asia-Pacific Education Researcher, 29(3), 285-296.
Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23-34.
Hyland, K. (2003). Second language writing. Cambridge University Press.
Hyland, K., & Hyland, F. (2006). Feedback in second language writing: Contexts and issues. Cambridge University Press.
IELTS. (2020). IELTS Writing band descriptors (public version).
Johnson, R. B., & Christensen, L. (2012). Educational research: Quantitative, qualitative, and mixed approaches (5th ed.). SAGE Publications.
Kukulska-Hulme, A., & Shield, L. (2018). An overview of intelligent computer-assisted language learning. In S. Thouësny & L. Bradley (Eds.), Calling on CALL: From theory and research to new directions in foreign language teaching (pp. 381-399). Routledge.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174.
Lee, I. (2019). The power of feedback in second language writing. Journal of Second Language Writing, 45, 1-13.
Pallant, J. (2013). SPSS survival manual: A step-by-step guide to data analysis using IBM SPSS (6th ed.). Open University Press.
Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). SAGE Publications.
Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on EFL learners’ writing accuracy. Computer Assisted Language Learning, 27(3), 248-262.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Wang, Z. (2020). Computer-assisted EFL writing and evaluations based on artificial intelligence: A case from a college reading and writing course. Library Hi Tech, 38(1), 172-193.
Ware, P., & Warschauer, M. (2006). Electronic feedback and second language writing. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 105-122). Cambridge University Press.
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). A systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1-27.
