RANA JAHANI
UX Researcher | Game User Researcher | Data Analyst
Checking Design Considerations for Reflection in XR Learning with Real Users
About Project: As part of a collaborative research team, we designed and tested an extended reality learning environment (XRLE) that teaches logical thinking through puzzle-based gameplay. We adapted reflection strategies from escape room design into concrete game mechanics and built them into a custom XR experience. Through a mixed methods user study with 14 players, combining gameplay observation and interviews, we found both strengths and challenges in how players responded to reflective prompts. From these results, we refined design guidelines for integrating reflection into immersive games, supporting stronger engagement and learning outcomes.
My Role: UX Researcher, Game Designer, Data Analyst
My Responsibilities:
►Development:
-
Designed and developed a virtual escape room in Unity, enhancing problem-solving skills through immersive puzzle-solving experiences for PC, VR, and AR platforms
-
Prototyped game mechanics, levels, and puzzles, using C# to create engaging player experiences in an iterative process
►User Research:
-
Conducted qualitative research to gather and analyze data on user behavior and preferences
-
Organized and facilitated user research studies, including recruiting participants, holding playtesting sessions, conducting interviews and surveys to gather user data
-
Utilized qualitative methodologies such as open coding to identify themes, patterns, trends, and anomalies in user behaviors to enhance reflection and hint system in extended reality learning environments
-
Communicated research findings across teams to guide product development and design decisions
-
Collaborated with the user research team, designers, and engineers to support the research plans and iterate on the product development
-
Co-authored the research paper to reflect research findings and advance industry and academic knowledge
RESEARCH METHODS
-
Research Question: Reflection is a tried and true metacognitive learning support with a long history of success in helping students get more out of educational experiences. In this work, we examined how reflective prompts are experienced by learners in extended reality learning environments.
-
Experiment Design: To explore our idea, we developed a VR escape room using Unity, aimed at enhancing logical thinking and problem-solving skills through immersive and interactive puzzle-solving. Within the narrative of the experience, players are exploring an abandoned house, where they are able to access two rooms: a kitchen and a greenhouse. Within this environment, there are exactly five logic grid puzzles related to cooking and gardening that the player must solve
Escape room Layout
Escape room Flow
Escape room Puzzle Structure
Escape room Puzzles
RESEARCH METHODS
During the study while players were playing the game, researchers provided two types of aid: reflective prompts and explicit hints.
-
Data Analysis:
-
Following a combined observation and interview methodology, we recorded participants’ run-throughs of the VR escape room and interviewed them afterwards about their thoughts on and perceptions of the reflective prompts they received.
-
Interview Analysis: Using open coding method for qualitative analysis, we labeled each participant response to the interview and organized and consolidated their labels into initial codes. Then we generated a list of 8 themes from the initial codes.
-
We then coded the rest of the interview transcripts with these themes and then tested for validity via inter-rater reliability, again using Cohen’s Kappa.
-
Run-through Video Analysis: From the videos of the escape room run-throughs, we wanted to see how participants responded, and behaved in response, to the prompts. Therefore, we adapted the situational awareness (SA) model used to analyze reflections.
-
We applied one code to each reflective response spoken by a participant. In addition, the action taken after the response was coded as either a correct action, an incorrect action, disengagement, or no action. The SA model codes were checked for inter-rater reliability via Cohen’s Kappa.
-
Telemetry Data Analysis: We additionally collected and present descriptive statistics related to time
taken to complete the room and individual puzzles and number of hints received by participants.
-
An image of the room where the study was held. Two researchers sat at a table facing an open space where the participant could move freely. The image is obscured to protect participant identity and maintain anonymity. In the corner is an image of what the participant saw in the headset.
Eight themes emerged from the interviews, which we organize into three higher-level categories based on common concepts. The table additionally presents the number of times each theme appeared in the data (column 1) and the number of participants who
mentioned it (column 2).
An overview of the reflection response and next action trajectories
OUTCOMES
-
We found that, while the learners’ perceptions validated parts of the approach, additional considerations are needed to account for how players perceive and experience the prompts on the receiving end. From these results, we expanded upon the theoretical foundations presented in prior work by incorporating insights into how users perceive and experience these reflective prompts in
extended reality learning environments and provide refined considerations for future use.