REFINE: Real-world Exploration of Interactive Feedback and Student Behaviour

View PDF HTML (experimental)

Abstract:Formative feedback is central to effective learning, yet providing timely, individualised feedback at scale remains a persistent challenge. While recent work has explored the use of large language models (LLMs) to automate feedback, most existing systems still conceptualise feedback as a static, one-way artifact, offering limited support for interpretation, clarification, or follow-up. In this work, we introduce REFINE, a locally deployable, multi-agent feedback system built on small, open-source LLMs that treats feedback as an interactive process. REFINE combines a pedagogically-grounded feedback generation agent with an LLM-as-a-judge-guided regeneration loop using a human-aligned judge, and a self-reflective tool-calling interactive agent that supports student follow-up questions with context-aware, actionable responses. We evaluate REFINE through controlled experiments and an authentic classroom deployment in an undergraduate computer science course. Automatic evaluations show that judge-guided regeneration significantly improves feedback quality, and that the interactive agent produces efficient, high-quality responses comparable to a state-of-the-art closed-source model. Analysis of real student interactions further reveals distinct engagement patterns and indicates that system-generated feedback systematically steers subsequent student inquiry. Our findings demonstrate the feasibility and effectiveness of multi-agent, tool-augmented feedback systems for scalable, interactive feedback.

Comments: Accepted to AIED 2026

Subjects:

Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)

Cite as: arXiv:2603.29142 [cs.AI]

(or arXiv:2603.29142v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2603.29142

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Fares Fawzi [view email] [v1] Tue, 31 Mar 2026 01:48:08 UTC (1,382 KB)