Using Automated Within-Subject Invisible Experiments to Test the Effectiveness of Automated Vocabulary Assistance

Gregory Aist and Jack Mostow
Proceedings of ITS'2000 Workshop on Applying Machine Learning to ITS Design/Construction, Fifth International Conference on Intelligent Tutoring Systems, July, 2000, pp. 4 - 8.


Abstract
Machine learning offers the potential to allow an intelligent tutoring system to learn effective tutoring strategies. A necessary prerequisite to learning an effective strategy is being able to automatically test a strategy's effectiveness. We conducted an automated, within-subject "invisible experiment" to test the effectiveness of a particular form of vocabulary instruction in a Reading Tutor that listens. Both conditions were in the context of assisted oral reading with the computer. The control condition was encountering a word in a story. The experimental condition was first reading a short automatically generated "factoid" about the word, such as "cheetah can be a kind of cat. Is it here?" and then reading the sentence from the story containing the target word. The initial analysis revealed no significant difference between the conditions. Further inspection revealed that sometimes students benefited from receiving help on "hard" or infrequent words. Designing, implementing, and analyzing this experiment shed light not only on the particular vocabulary help tested, but also on the machine-learning-inspired methodology we used to test the effectiveness of this tutorial action.

Notes
Associated Lab(s) / Group(s): Project LISTEN
Associated Project(s): Project LISTEN\'s Reading Tutor

Text Reference
Gregory Aist and Jack Mostow, "Using Automated Within-Subject Invisible Experiments to Test the Effectiveness of Automated Vocabulary Assistance," Proceedings of ITS'2000 Workshop on Applying Machine Learning to ITS Design/Construction, Fifth International Conference on Intelligent Tutoring Systems, July, 2000, pp. 4 - 8.

BibTeX Reference
@inproceedings{Aist_2000_3679,
   author = "Gregory Aist and Jack Mostow",
   editor = "Joseph Beck",
   title = "Using Automated Within-Subject Invisible Experiments to Test the Effectiveness of Automated Vocabulary Assistance",
   booktitle = "Proceedings of ITS'2000 Workshop on Applying Machine Learning to ITS Design/Construction, Fifth International Conference on Intelligent Tutoring Systems",
   pages = "4 - 8",
   month = "July",
   year = "2000",
}