Language models in ELE: comparison of ChatGPT, LLaMA 3 and Gemini for levelling and feedback on written expression

Abstract

The incorporation of large language models has opened up new possibilities for teaching Spanish as a foreign language, especially in relation to personalised learning. In this context, the levelling of written expression and formative feedback are key areas for supporting both the assessment and revision of texts produced by learners. The aim of this study is to compare the performance of ChatGPT, LLaMA 3 and Gemini in levelling and feedback tasks for written expression in Spanish language. The methodology is based on the analysis of texts written by learners from the Corpus de Aprendices de Español (Corpus of Spanish Learners), to which the same structured prompt was applied, aligned with the descriptors of the Common European Framework of Reference for Languages and the Instituto Cervantes Curriculum Plan. The results show different profiles among the models in terms of accuracy, curricular adequacy, and quality of feedback, which allows for an assessment of their potential and limitations for pedagogical integration based on the teaching of Spanish as a foreign language.