January 23, 2007: Seminar: Ryan S. J. d. Baker: "Detecting and Adapting to When Students Game the System "
Detecting and Adapting to When Students Game the System
Ryan S. J. d. Baker
University of Nottingham
Tuesday, January 23, 2007
3:00 p.m., Room 1000 SEO
Students use intelligent tutors and other types of interactive learning environments in a considerable variety of ways. In this talk, I will present research on automatically detecting and
adapting to when students "game the system", attempting to succeed in a learning environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly.
I will present a set of studies that establish that gaming the system is replicably associated with low learning, and will present evidence on which motivations, attitudes, and affective states are associated
with the choice to game the system. I will also discuss evidence that the relationship between gaming and learning differs, depending on when and why a student chooses to game.
I will then present a detector that reliably detects gaming, in order to drive adaptive support. In order to make accurate predictions at multiple grain-sizes -- predicting both which students game, and when a specific student is gaming -- the detector was trained using a psychometric modeling framework, Latent Response Models (Maris, 1995) in combination with a machine-learning space-searching technique, Fast Correlation-Based Filtering (Yu and Liu, 2003). My colleagues and I have
validated that this detector transfers effectively between several intelligent tutor lessons without re-training, despite the lessons varying considerably in their subject matter and user interfaces.
The gaming detector has been used to develop a tutor lesson which responds to gaming. Within this lesson, a software agent ("Scooter the Tutor") indicates to the student and their teacher whether the student has been gaming recently. Scooter also gives students supplemental exercises, in order to offer the student a second chance to learn the material he/she had gamed through. Scooter reduces the frequency of gaming by over half, and Scooter's supplementary exercises are associated with substantially better learning; Scooter appears to have had virtually no effect on students who do not game.
Ryan S. J. d. Baker is a Research Fellow in the Learning Sciences Research Insitute at the University of Nottingham. He graduated from Carnegie Mellon University's School of Computer Science in December 2005, with a Ph.D. in Human-Computer Interaction. His research
interests are at the intersection of human-computer interaction, educational data
mining, educational psychology, and machine learning. His long-term goal is to develop a general framework for learner-computer interaction, which will detail the variety of ways students choose to interact with interactive learning environments, what factors lead students to respond to the same
learning environment in different ways, and how interactive learning environments can appropriately and effectively adapt to the relevant differences in student behavior.
Host: Tom Moher