AI: Summary
This featured discussions of the potential of XR for visualizing academic content, particularly how to create meaningful spatial representations of knowledge that go beyond conventional 2D interfaces. Key themes included creating “wow” moments in XR interfaces, balancing simplicity with power, and how to meaningfully use the third dimension to represent knowledge. Participants shared ideas about using metaphors like debug sticks, incorporating non-linear spatial arrangements, and viewing academic papers in multiple dimensions.
Frode Hegland, Fabien Bénétou, Mark Anderson, Peter Wasilko, Karl Arthur Smink, Tom Haymes
AI: Speaker Summary
Frode Hegland focused on developing gesture grammar for XR interfaces and exploring “wow” moments that could transform how users interact with knowledge. He mentioned meeting with Dene, Emily and Edgar, and emphasized the need to approach gesture design from multiple angles simultaneously, even though this process can be stressful. Hegland suggested developing scenarios for interaction, such as what a “secret agent” might do, and proposed features like glossaries that could be expanded spatially. He was interested in creating views that animate between different spatial arrangements and thinking about dimensions beyond the typical keyword searches.
Fabien Bénétou expressed concern about gesture design becoming too complex or requiring users to learn too many isolated commands. He emphasized the importance of creating composable gestures that can be combined to create complex interactions, comparing this to grammar or “Lego bricks” that can be assembled. Bénétou suggested focusing on functions that apply to multiple items simultaneously (like sorting) and finding ways to make gestures scale from single items to groups. He introduced the concept of “eidetic gestures” – empty containers that gain meaning when applied to specific objects.
Mark Anderson raised concerns about making tools too exclusive or expert-focused, questioned how to make “wow” moments generalizable, and emphasized the importance of being “in” the visualization as a key XR advantage. He suggested developing a timeline for the Future of Text book as a practical project. Anderson also noted that for academic papers to support multiple views, the content needs to support those views with appropriate data. He was interested in the connection between papers and their underlying data, especially in non-humanities fields.
Peter Wasilko suggested looking at Japanese grammar for gestural inspiration, sharing a link about Japanese proximity-based grammar. He also shared a link to a resource on threaded interpretive languages. Wasilko made a reference to Blade Runner with his “origami unicorn” comment when discussing artifact creation in XR spaces.
Karl Arthur Smink shared a graph showing the relationship between ease of use and power in tools, noting that more complex tools become more powerful but harder to learn. He raised questions about false positives versus false negatives in gesture recognition and shared examples from gaming interfaces like Minecraft’s Debug Stick and Metroid Prime’s 3D map system. Smink emphasized the importance of contextual information and toggled views in 3D modeling software.
Tom Haymes focused on how XR could transcend physical world limitations, suggesting that the third dimension could represent concepts beyond physical depth. He shared resources on world-building and comics in digital space, emphasized the need to be both consumers and creators, and suggested automating the process of finding relevant articles during discussions. Haymes also shared an example of how he had mapped connections between his writings and influential works, noting it was labor-intensive but could potentially be automated.
AI: Topics Discussed
WebXR: Fabien mentioned Babia XR, sharing the URL https://babiaxr.gitlab.io and discussing visualization frameworks for WebXR. He also linked to researcher Jesús González Barahona (https://gsyc.urjc.es/jgb/) who appears to be connected to the BabiaXR project.
Gestures: Extensive discussion of gesture design principles for XR, emphasizing the need for composable gestures that form a coherent grammar rather than isolated commands. Fabien compared ideal gestures to Lego bricks that can be combined. Frode emphasized designing from multiple angles simultaneously. The concept of “eidetic gestures” was introduced – empty containers that gain meaning when applied to objects. Participants discussed categories of gestures (hand-based, wrist-based, two-handed) and time-bound gestures that only have validity in certain contexts.
Book: Mark Anderson suggested developing a timeline visualization for the Future of Text book as a practical XR project. He mentioned that the data already exists in JSON format, particularly the timeline appendix from the 2023 book.
Visual-Meta: Frode briefly mentioned Visual-Meta when discussing the possibility of writing a macro for Microsoft Word that would create visual metadata from documents for export to PDF, ensuring that position information from maps could be encoded into JSON and PDFs.
Timelines: Mark Anderson suggested implementing a timeline visualization for the Future of Text book as a practical project, noting that the data already exists in JSON format. He questioned what was holding the group back from implementing this feature that has been repeatedly discussed. Karl also mentioned timelines as a possible dimension for organizing cards of items based on publication dates.
Other topics discussed:
- The importance of context in XR experiences
- Balancing complexity and power in interface design
- Using metaphors from gaming like Minecraft’s Debug Stick
- The advantages of being “inside” a 3D visualization
- Multiple views of academic papers and their data
- World-building principles for XR spaces
- Political concerns about global situations (mentioned briefly at the end)
- Spatial hypertext and the Model Context Protocol
- The concept of “wow” moments in XR interfaces
- Creating flexible, responsive academic publishing formats
AI: Concepts Introduced
Eidetic gestures – Introduced by Fabien Bénétou as gestures that are “empty” until applied to something, functioning as containers that gain meaning from their context or application, similar to deictic words in language.
Gesture grammar – Discussed by both Frode and Fabien as a structured system of gestures that can be combined compositionally rather than learning isolated commands, similar to how language grammar works.
Debug Stick – Karl Arthur Smink introduced this concept from Minecraft, describing it as a tool that allows users to edit metadata of objects, only showing one editable property at a time to avoid overwhelming the user. The metadata is only visible and editable in the context of holding the stick.
Proximity-based grammar – Peter Wasilko suggested looking at Japanese grammar for gestural inspiration, specifically the use of words indicating relative distance between speaker and objects.
“Wow” moments – Frode Hegland and others discussed creating transformative experiences in XR that would impress users, focusing on the view changes rather than just the gestures themselves.
Responsive academic publishing – Mentioned by Fabien as a concept combining paper, poster, code, data, and setup environment in an integrated way.
AI: Notes for Claude
Mark Anderson asked: “Do you have a picture of a timeline Claude? Just pulling lates into a list is’t what I mean as dates can apply to different aspects of the whole in a way the alters the viewers experience.”
AI: People Mentioned
Bob Horn – Mentioned by Frode Hegland and Tom Haymes, referring to an email thread about layers in maps and the notion of selecting what to see. Tom specifically recommended looking at Bob Horn’s 2D work.
Dene – Mentioned by Frode Hegland several times, referring to Dene Grigar, the Co-PI on the project. Frode mentioned having a meeting with her the day before.
Edgar – Mentioned by Frode Hegland as his son who fell off kitchen counter and seemed to be okay, though it was concerning.
Scott McCloud – Mentioned by Tom Haymes who shared a link to his TED talk about comics’ transition to digital metaphors, noting it’s still relevant nearly 20 years later.
Adam – Mentioned by Frode Hegland as someone they hoped to have a session with.
Ismail – Mentioned by Frode Hegland as someone who commented on the notion of layers in Bob Horn’s email thread.
Tom Cruise – Referenced by Peter Wasilko as a joke about impressive gestures in movies.
Jesús González Barahona – Referenced indirectly by Fabien through a shared URL (https://gsyc.urjc.es/jgb/), apparently connected to the BabiaXR project.
AI: Product or Company Names Mentioned
Microsoft Word – Mentioned by Frode Hegland when discussing potentially creating a macro to generate Visual-Meta data for documents.
PDF – Mentioned by Frode Hegland in context of exporting documents with Visual-Meta data from Word.
JSON – Mentioned by Frode Hegland regarding encoding position information, and by Mark Anderson regarding existing data from the Future of Text book.
Excel – Mentioned by Frode Hegland when discussing viewing glossaries and transforming them into spatial arrangements.
Minecraft – Mentioned by Karl Arthur Smink when discussing the Debug Stick feature that allows contextual editing of object properties.
Maya (Autodesk Maya) – Mentioned by Karl Arthur Smink when sharing an example of a 4-view mode in 3D modeling software.
Metroid Prime – Mentioned by Karl Arthur Smink when describing a well-designed 3D map system in gaming.
Illustrator – Mentioned by Frode Hegland when noting he would need to redo the first Future of Text book in Author instead of Illustrator.
Author – Mentioned by Frode Hegland as the tool he would need to use instead of Illustrator to redo the first Future of Text book.
Science museum – Mentioned by Frode Hegland as a place he visited with Emily, Edgar and Dene that Fabien and his family might enjoy when visiting.
Photoshop – Mentioned by Karl Arthur Smink when comparing it to Paint as an example of complexity versus ease of use.
Paint – Mentioned by Karl Arthur Smink when comparing it to Photoshop as an example of simplicity versus power.
Model Context Protocol (MCP) – Shared by Mark Anderson as an interesting local link for AI and visualization in the context of spatial hypertext.
Museum of Fine Arts Houston – Mentioned by Tom Haymes when sharing an image of their infinity room.
Google Map – Mentioned by Tom Haymes as a metaphor for helping users find their way back in spatial interfaces.
AI: Agreements & Disagreements
Agreements:
- Frode and Fabien agreed that gestures should form a coherent grammar rather than isolated commands.
- Frode and Fabien agreed that creating “wow” moments should focus on the view changes rather than just the gestures themselves.
- Frode agreed with Fabien’s concern about requiring users to learn too many disconnected gestures.
- Multiple participants agreed on the importance of context in XR experiences.
- Several participants agreed that the ability to be “inside” visualizations is a key advantage of XR.
- General agreement that a balance between power and simplicity is important in interface design.
Disagreements: No significant disagreements were evident in the transcript. There were different emphases and perspectives, but no clear conflicts or opposing viewpoints.
Potential tension points (not explicit disagreements):
- Different emphases on how to approach gesture design – Frode suggested working from multiple angles simultaneously while Fabien focused more on composability.
- Varying perspectives on what constitutes a “wow” moment in XR interfaces.
- Different views on the appropriate level of complexity for XR interfaces.
AI: Other
The transcript reveals a group that is actively exploring new paradigms for knowledge representation in XR while drawing inspiration from many different fields, including gaming, comics, Japanese language, and traditional academic publishing. Their approach combines theoretical discussions with practical examples and personal experiences.
Chat Log URLs
https://systemsorienteddesign.net/master-thesis-beyond-labels/ https://archive.org/details/R.G.LoeligerThreadedInterpretiveLanguagesTheirDesignAndImplementationByteBooks1981 https://cdn8.openculture.com/wp-content/uploads/2015/09/11203454/biggest-book-in-the-world.jpg https://ideaspaces.net/unflattening-ideas/ https://minecraft.fandom.com/wiki/Debug_Stick https://www.ted.com/talks/scott_mccloud_the_visual_magic_of_comics/transcript https://rosenfeldmedia.com/books/make-it-so/ https://jfsdigital.org/wp-content/uploads/2015/12/01_Articles02_What-in-the-World-2.pdf https://babiaxr.gitlab.io https://gsyc.urjc.es/jgb/ https://miro.com/app/board/uXjVP2DSvu4=/?share_link_id=716747414294 https://modelcontextprotocol.io/introduction https://futuretextlab.info/2025-schedule/
This chat log documents a lively discussion among participants about XR interfaces for academic and knowledge work. Key themes included creating 3D visualizations that go beyond traditional 2D interfaces, developing gesture-based controls, and exploring metaphors from various fields. Participants shared resources on world-building, comics in digital spaces, visualization frameworks, and the Model Context Protocol. Karl Arthur Smink shared a graph showing the relationship between ease of use and power in tools, while Tom Haymes emphasized how XR could transcend physical limitations. Several participants shared examples from gaming interfaces like Minecraft’s Debug Stick and Metroid Prime’s 3D map system. Mark Anderson suggested developing a timeline visualization for the Future of Text book as a practical project. The discussion included philosophical considerations about spatial interfaces and the need for interfaces that support both expert and novice users.