Abstract:

Many older adults, particularly those with Mild Cognitive Impairments (MCI) struggle with complex, sequential tasks such as meal preparation. In this thesis, we present a framework for personalized sequence prediction and assistance detection during meal preparation to support older adults, particularly those with Mild Cognitive Impairments (MCI). By leveraging the reasoning capabilities of large language models (LLMs), our system anticipates user actions and identifies moments when assistance may be needed. We introduce two methods for preference-based sequence prediction, called Independent Context and Shared context, using either a participant’s own prior actions or sequences from others as context. Evaluated on two meal preparation datasets, these approaches outperform baseline models by up to 33.8%, demonstrating both user-specific adaptation and generalization across cooking domains. To inform assistance strategies, we conducted a meal preparation data-collection study with older adults at two independent living facilities. Insights from this study revealed common errors, such as forgotten items or visits to irrelevant locations. We used these findings to develop a second approach that detects such mistakes and prompts assistance using LLM reasoning. This approach was validated on both synthetic and real-world data, showing strong performance in identifying when users may need help. These contributions form the basis of a personalized assistive system that supports users while preserving their independence in daily meal preparation tasks.

Committee:
Reid Simmons, co-chair
Zackory Erickson, co-chair
Illah Nourbakhsh
Patrick Callaghan