AI: Summary
This meeting focused on “Active Memory Palaces and XR,” moderated by Frode Hegland. The discussion examined how traditional memory palace techniques could be reimagined for Extended Reality environments. Participants explored the concept of spatial information organization, moving beyond traditional memory palaces toward “active information spaces” that leverage human spatial cognition for knowledge work. The conversation covered practical implementations, user interactions, and the potential for XR environments to serve as persistent workspaces where information maintains spatial relationships. Fabien Bénétou demonstrated actual XR implementations including persistent storage across sessions and visual citation systems. The group discussed the challenges of designing intuitive spatial interfaces and the importance of maintaining both stability and flexibility in information organization.
A strategy for memory enhancement, which uses visualizations of familiar spatial environments in order to enhance the recall of information. wikipedia
We ask how we can build Memory Palaces in XR and how can we make them active and interactive. We hope that this discussion can inspire articles on this topic for The Future of Text 6. If you have used Memory Palaces or if you have a specific perspective/something to demo, please get in touch to tell us and we will schedule some time for you up front.
We will ask, when your work can be everywhere, how best to control it?
- In the first place we might ask*: How does other Mappings & Shapes of Knowledge relate to Memory Palaces?
- Find How can they help us find information?
- Thought How can they augment thought?
- Express Can such Memory Palaces also be effective means of expression
- Learn How can they help us learn?
- VR & AR How would Memory Palaces be different in VR and AR? How might we transition between the two?
- Multiple Palaces How can they work in the physical rooms you work in, such as home and office, and not just one?
- Change How can they instantly change to a different project–and how should they? What interactions can be suitable for this change?
- Connect & Nest How can they be connected and nested? How can relevant volumes of information be available without being overwhelming?
- Access Levels How might semantic zooming help us better understand our work at different levels since too much detail can obscure other patterns and shapes of information.
- Framing How can Memory Palaces encourage, discourage and frame specific ways of looking and thinking–how can it frame perspectives?
What superpowers should we have in our memory places in XR?
Frode Hegland, Ayaskant Panigrahi, Ken Perlin, Peter Wasilko, Mark Anderson, Fabien Bénétou, Jim Strahorn, Karthik Agaram, Jonathan Finn, Rob Swigart, Peter Dimitrios, Brandel Zachernuk
AI: Speaker Summary
Frode Hegland served as the moderator and presenter, introducing the concept of “Active Memory Palaces and XR” through a brief slideshow. He emphasized the need to move beyond traditional memory palace concepts toward spatial information environments in XR. Hegland was particularly interested in specific interaction design questions – how users would select, organize, and manipulate information objects in 3D space. He advocated for creating inspiring demonstrations that could encourage students and others to engage more deeply with their knowledge work, comparing it to Iron Man-style interactions.
Ayaskant Panigrahi brought expertise in 3D user interfaces and gestural research. He connected memory palaces to his research background and discussed the importance of micro-gestures for frequent operations without fatigue. He emphasized the difference between recall and recognition in memory processes, suggesting that having visual prompts available is easier than trying to recall from a blank state. Panigrahi also noted how spatial terms have already been adopted for web navigation and connected the discussion to inner monologue research.
Ken Perlin provided historical context as co-creator of semantic zooming and emphasized the fundamental shift XR represents. He described his vision of moving from “workspace” to “workshop” – environments where people use physical space naturally for information work, similar to how movie productions use walls and models. Perlin positioned XR as a post-singularity technology that will make screens obsolete, allowing humans to leverage their evolved spatial cognition superpowers with AI assistance.
Peter Wasilko brought extensive knowledge of traditional memory palace techniques and hypertext research. He discussed the dual nature of memory palaces – both as mnemonic devices and as finding aids in technological contexts. He was particularly interested in spawning multiple copies of information objects in different thematic areas and using concrete objects as anchors for abstract data. Wasilko also raised concerns about declining memory skills in education and the need for higher expectations and better training methods.
Mark Anderson focused on the challenges of sharing rich spatial information between users. He questioned whether users need to see the full complexity of others’ memory spaces or just relevant contextual information. Anderson emphasized the importance of surfacing contingent ideas rather than just more of the same, relating this to discussions of informational Level of Detail (LoD).
Fabien Bénétou provided the technical implementation perspective, demonstrating actual working XR systems with persistent storage, visual citations, and document manipulation. He was somewhat critical of traditional memory palace techniques, viewing them as obsolete given modern technology, but strongly supported the spatial organization of information concepts. Bénétou emphasized the importance of web-based XR solutions over proprietary apps and showed practical examples of his work.
Jonathan Finn offered analytical perspectives on the strict versus loose definitions of memory palaces, distinguishing between map-like precision and exploration-based navigation. He proposed that initial XR views should show probabilistic predictions of what users might need next, arranged by likelihood but maintaining familiar spatial layouts. Finn also shared fascinating examples of synesthesia as natural spatial memory systems.
Karthik Agaram drew parallels between memory palaces and programming environments, describing how spatial arrangements in his development tools became persistent mental anchors. He emphasized the importance of having blank spaces for context clearing and related memory palace concepts to broader questions about human tolerance for forgetfulness versus the need for quick context switching.
Rob Swigart provided clarification on traditional memory palace purposes – as tools for memory aid rather than text creation. He appreciated the spatial organization aspects while noting the distinction between classical techniques and proposed XR applications. Swigart also contributed humor about screens proliferating like bunnies in modern environments.
Peter Dimitrios focused on the practical aspects of authoring for different audiences – personal future reference versus sharing with others. He discussed the challenges of making XR comfortable for text-heavy work and emphasized the continued need for search functionality alongside spatial organization. He supported skeuomorphic approaches for accessibility while acknowledging the need for more abstract capabilities.
Brandel Zachernuk shared his experience building Word Reality, a VR word processor, highlighting the potential for combining verbal input with spatial information organization. He discussed the value of having search results appear in peripheral space while maintaining focus on primary tasks, and the potential for gestural interfaces to create composite views of information.
AI: Topics Discussed
WebXR Fabien Bénétou strongly advocated for web-based XR solutions over proprietary apps, emphasizing the importance of avoiding “information ghettos” and maintaining long-term accessibility. He demonstrated actual WebXR implementations including persistent storage using browser APIs, document manipulation systems, and visual citation tools. The discussion included technical details about local versus remote storage options and the advantages of web-based systems for knowledge management. Bénétou also mentioned collaboration with Leon on CLI interfaces in WebXR.
Gestures Ayaskant Panigrahi led discussion on gestural interfaces, emphasizing the importance of micro-gestures for frequent operations without user fatigue. He referenced Google’s Soli project and Meta’s SDK implementations of micro-gestures. The conversation covered the challenge of designing expressive yet sustainable gesture vocabularies, comparing them to keyboard shortcuts in traditional interfaces. Frode Hegland suggested specific interaction ideas like pinch-and-pull for layout options and twist gestures for parameter control, advocating for muscle memory development similar to driving controls.
Other topics discussed included: Traditional memory palace techniques and their historical context Semantic zooming and its applications in XR environments
The “doorway effect” and context switching in spatial environments Educational implications and declining memory skills Synesthesia as a natural spatial memory system The difference between workspace and workshop metaphors Skeuomorphic versus abstract interface design Persistence and session management in XR environments Visual citation systems and information tracking The kitchen mise-en-place concept as a spatial cognition model
Interesting anecdotes included: Ken Perlin’s account of reaching for books in his office based on spatial memory Jonathan Finn’s friend with strong synesthesia who organizes dates in circular patterns and has a unique number line visualization Mark Twain hanging cards on trees while walking in parks Frode Hegland’s calendar year visualization as a clock-like circle Peter Wasilko seeing Jeremy Brett perform Sherlock Holmes on stage in London The discussion of how spatial terms naturally emerged for web navigation
AI: Concepts Introduced
Active Memory Palaces – Frode Hegland introduced this as an evolution beyond traditional memory palace techniques, focusing on spatial information environments in XR rather than rote memorization
Active Information Spaces – The group collectively moved toward this term as a better descriptor than “memory palaces” for their XR-based spatial information systems
Workshop vs Workspace – Ken Perlin distinguished between traditional “workspace” metaphors (like Meta’s office simulations) and his preferred “workshop” concept where people use space naturally for creative and productive work
Mise-en-place for Information – Fabien Bénétou applied the kitchen preparation concept to information organization, where spatial arrangement supports cognitive processes
Information Ghettos – Frode Hegland’s term for isolated, incompatible information systems that prevent fluid knowledge work
Semantic Zooming – Ken Perlin provided historical context for this concept he co-created, relating it to information visibility at different scales
Spatial Tags – Peter Dimitrios proposed this as an alternative to textual labeling systems
Memory Spaces – Used interchangeably with memory palaces but emphasizing the spatial rather than mnemonic aspects
AI: People Mentioned
Doug Engelbart, mentioned by Peter Wasilko in context of co-evolution of technology and users; Dave (presumably David, colleague of Doug Engelbart); David Fox, Ken Perlin’s former student who co-created semantic zooming; Ben Peterson, who built upon the Pad semantic zooming work; Giordano Bruno, mentioned by Rob Swigart as author of memory palace books; Mark Twain, referenced for his use of cards on trees as memory aids; Tim Cook, mentioned by Frode Hegland regarding factory location discussions; Jeremy Brett, praised by Kartik Agaram and Peter Wasilko as definitive Sherlock Holmes actor; Bob Horn, mentioned by Frode Hegland for his emphasis on lists and organization; Sherlock Holmes (fictional), referenced multiple times as memory palace inspiration; Leon van Kammen, mentioned by Fabien Bénétou as WebXR CLI collaborator; Alex (Jonathan Finn’s friend), described as having strong synesthesia for dates and numbers; A.R. Luria, mentioned by Jonathan Finn as author of “The Mind of a Mnemonist”; Oliver Sacks, noted as inspired by Luria’s work; Chris (Gebhardt), mentioned by Frode Hegland regarding document concepts; Dene Grigar, referenced for future discussion of framing perspectives; Edgar, mentioned as Frode’s son attending a Jesuit school.
AI: Product or Company Names Mentioned
Meta, mentioned multiple times regarding office workspace videos, SDK micro-gestures, and VR headsets; Facebook, mentioned as Meta’s previous name and social media platform; Google, referenced for Soli micro-gesture project and Google Brain (Ken Perlin’s background); Microsoft, mentioned regarding Office suite dominance and workspace metaphors; OpenAI, referenced as Ken Perlin’s previous employer where he led GPT development; Anthropic, mentioned as maker of Claude AI; HTC Vive, mentioned by Brandel Zachernuk for his Word Reality project; Quest 2/Quest 3, discussed as VR headsets used by Fabien and others; Vision Pro, mentioned as high-end but potentially overkill VR option; WebXR, extensively discussed as preferred platform; Steam Deck, mentioned by Fabien as portable development device; Steam Link, used by Fabien for wireless XR development; Anki, mentioned by Peter Wasilko for spaced repetition learning; Instagram, mentioned by Frode regarding social media outreach; Blue Sky and X (Twitter), mentioned for meeting promotion; ACM Hypertext conference, referenced regarding Mark Anderson’s bibliography work; BBC, mentioned for Sherlock TV adaptation; Sloan Foundation, referenced as project funder; Iron Man, mentioned as inspiration for interaction design; YouTube, referenced for meeting recordings
AI: Other
The meeting revealed a fascinating tension between respecting traditional memory palace techniques while pushing toward more ambitious spatial computing applications. The group demonstrated remarkable technical depth, with working implementations being shown alongside theoretical discussions. There was a strong emphasis on avoiding the mistakes of previous proprietary systems and maintaining open, web-based approaches. The educational implications were particularly noteworthy, with concerns about declining memory skills balanced against excitement about new learning possibilities. The discussion successfully moved from abstract concepts to specific interaction design questions, setting up concrete areas for future development work.
Chat Log URLs
https://doi.org/10.1145/166117.166125 https://www.youtube.com/watch?v=co5rbkyRXOo https://apps.ankiweb.net https://fabien.benetou.fr/Wiki/VirtualRealityInterface https://www.dropbox.com/scl/fi/w9x92417618kuixohs2sk/Memory-Palaces-14th-July.pdf?rlkey=u50zd3rv87gdcgjk6ici126ww&dl=0 https://video.benetou.fr/videos/local https://companion.benetou.fr/index.html?username=q3_annotatedbibliography&remoteStorageURL=/q3_annotatedbibliography_savingDataDemo_q3_annotatedbibliography.json https://en.wikipedia.org/wiki/Doorway_effect https://www.persee.fr/doc/reso_0751-7971_1990_num_8_43_1783 https://xrsh.isvery.ninja/ https://fabien.benetou.fr/Languages/OwnConcepts#ResponsiveProgramming https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XR_META_hand_tracking_microgestures https://bsky.app/profile/cabanier.bsky.social/post/3looyrh5gnc2s https://futuretextlab.info/2025/07/05/ayaskant-panigrahi-july-21st/ https://futuretextlab.info/2025-schedule/
Chat Log Summary
The chat log reveals active parallel conversation during the meeting, with participants sharing relevant links and expanding on verbal discussions. Key contributions included Mark Anderson providing the citation for Ken Perlin’s foundational semantic zooming paper, Fabien sharing multiple technical demonstrations and philosophical perspectives on memory techniques, and Jonathan Finn recommending A.R. Luria’s work on exceptional memory. The chat showed strong engagement with participants reacting to comments, sharing technical resources, and providing additional context. Fabien was particularly active in sharing implementation links and discussing the tension between inspiration and practical reality. The chat also revealed technical discussions about VR hardware capabilities and comfort for text work, with Peter Dimitrios noting the challenges of coding in headsets. The conversation maintained the meeting’s themes while adding depth through shared resources and real-time commentary.
Work shown
The work shown by Fabien is available here: https://video.benetou.fr/videos/local
Inspirations
Our current work on Authoring in XR:
Virtual Reality Memory Palaces in ‘Matrise’ by Joakim Vindenes:
SoftSpace, by Yiliu Shen-Burke (including AR and move in VR by making a ‘fist’) as presented to our group:
Fabien early experiments:
(* “in the first place” and so on are related to the concept of memory palaces)