← Back to Research
Enhancing GenAI–based Information Seeking with Metacognitive Cues
When we search with AI, we tend to accept the first answer we get. What if the tool itself nudged you to slow down and think more carefully using metacognitive cues?
Methods
Formative between-subjects Wizard-of-Oz study comparing search and learning with vs without metacognitive cues + Retrospective think aloud interviews to understand thought process underlying search behaviorN = 40 college students
Developed 3 measures of critical human-AI engagement based on users' interaction data: Persistent Inquiry, Source Engagement, and Independent Thinking
Built MetaCues, an interactive GenAI-powered search and note-taking tool which proactively generates metacognitive cues by monitoring users' chat history, notes, and clickstream data
RCT on Prolific comparing MetaCues against baseline system with no cues, measuring impact on search behavior, learning outcomes and informed judgementsN = 146 adult learners, ages 18–59 (median: 18–24)
Quantitative analysis of interaction data + GenAI-assisted qualitative analysis of user and AI responses to capture search behavior, engagement and reasoning processes
Key Findings
Metacognitive cues help users engage more actively and critically with AI-generated information — asking more questions, engaging with sources, evaluating AI responses, and considering overlooked perspectives.
Impact
- Led to 46% increase in number of search queries in GenAI search, demonstrating significantly more active engagement, by adding metacognitive cues to GenAI-based search.
- Increased user exploration breadth by 88% and improved follow-up questioning by 60%.
- Defined and operationalized measures capturing critical engagement using interaction data.
- Designed and developed MetaCues, an interactive system for GenAI-based search and learning and evaluated its impact on adult learners' search behavior, learning outcomes, and informed judgements.